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First  Plenary  Session 


Opening  Ceremony 


Emerging  Issues  in 
Public  Health 


MUX 


THE  QUINTESSENTIAL  ROLE  OF  INFORMATION  IN  PUBLIC  HEALTH 

Keynote  Address 

Roz  Diane  Lasker,  M.D. 

Deputy  Assistant  Secretary  for  Health  (Policy  Development) 

U.S.  Department  of  Health  and  Human  Services 


It  is  a  pleasure  and  a  privilege  to 
be  with  you  this  morning,  not  only  to 
highlight  the  guintessential  role  of 
information  in  public  health,  but  also 
to  recognize  the  important  roles  that 
many  of  the  participants  of  this  meeting 
are  playing  in  collecting  and  using 
health-related  data,  and  to  share  with 
you  activities  at  the  federal  level  that 
have  the  potential  to  reorient  data 
policy  in  support  of  population  health. 

As  all  of  you  are  aware,  this  is  an 
incredibly  challenging  time  for  those  of 
us  who  are  involved  with  and  care  about 
health  information.   Information  to 
support  the  public's  health  has  never 
before  been  so  important.   Yet  many 
forces,  including  ever  tightening  budget 
constraints,  are  making  it  increasingly 
difficult  to  successfully  make  the  case 
for  investing  in  data  collection  and 
analysis.   Partnerships  between  federal, 
state,  and  local  governments,  and 
between  the  public  and  private  sectors 
are  becoming  increasingly  necessary  to 
meet  community-wide  information  needs. 
Yet  it  is  difficult  to  promote  the 
cooperation  required  to  support  these 
partnerships  in  an  environment  in  which 
health  information  is  viewed  more  as  a 
competitive  weapon  than  as  a  tool  for 
the  public  good. 

This  morning,  I  would  like  to 
highlight  specific  areas  where  the 
varied  participants  of  this  conference 
can  work  together  to  enhance  the 
usefulness  of  health  information.   By 
the  end  of  my  talk,  I  hope  that  many  of 
you  will  be  encouraged  to  join  in  these 
activities.   More  important,  however,  I 
hope  that  many  of  you  will  become 
actively  involved  in  the  data  policy 
debate.   Ultimately,  the  availability  of 
information  to  support  population  health 
depends  as  much  on  policy  as  on 
technology  and  methodology.   So  far, 
policy  relating  to  health  data  has  been 
debated  primarily  by  those  representing 
the  business  side  of  health  care 
(purchasers  and  large  providers/payers) 
and  by  advocacy  groups.   Adding  to  the 
table  professionals  who  actively  collect 
and  analyze  data  for  broad-based  public 
health  purposes  can  bring  an  important 
perspective  to  bear  when  decisions  are 
being  made.   The  extent  to  which  each  of 
you  keeps  abreast  of  developments  and 
weighs  in  where  and  when  it  matters  will 
be  an  important  factor  determining  how 
policymakers  view  health  information  and 
how  that  information  is  collected  and 
used. 


THE  GROWING  IMPORTANCE  OF  HEALTH 
INFORMATION 

As  the  health  system  and  the  role 
of  government  undergo  rapid  changes,  the 
need  for  useful  information  is  becoming 
increasingly  important.   We  need 
information  to: 

-  track  the  impact  of  the  dramatic 
changes  that  are  occurring; 

-  measure  health  system  performance 
(this  is  particularly  important  as 
we  move  toward  a  smaller  and  less 
regulatory  role  for  government) ;  and 

-  provide  the  knowledge  base  that  we 
need  to  take  effective  action  and 
improve  performance  (for  example: 
Where  are  interventions  needed? 
Which  interventions  are  most  cost- 
effective?   What  is  the  outcome  when 
various  interventions  are 
implemented? ) 

For  those  involved  in  the  health 
care  system,  information  is  needed  about 
the  quality,  availability,  and  costs  of 
care,  and  to  support  consumer/purchaser 
choice  and  health  plan  report  cards. 
For  those  of  us  involved  in  public 
health  and  health  policy,  the 
perspective  and  need  for  information  is 
even  broader.   The  bottom-line  question 
here  is  "How  do  we  get  the  biggest  bang 
for  the  buck  in  health"?  And  although 
most  of  our  country's  attention  and 
investment  focuses  on  the  health  care 
system,  the  answer  to  this  question  does 
not  lie  in  the  medical  sector. 

As  most  of  you  know,  more  than  70 
percent  of  premature  deaths  in  this 
country  are  due  to  personal  risk 
behaviors  and  environmental  hazards  that 
the  medical  system  can  do  little  about. 
To  make  an  impact  on  health  and  reduce 
disparities  in  health  status,  we  need 
the  capability  to  provide  population- 
based  services  that  reduce  or  eliminate 
threats  to  health  in  the  community.   To 
do  this,  we  need  information  that  goes 
beyond  medical  care,  encompassing  the 
entire  population  (including  those  who 
do  not  seek  out  care) ,  the  environment, 
and  the  population-based  public  health 
system. 

RETHINKING  HEALTH  DATA  STRATEGY  IN  A 
CHANGING  ENVIRONMENT 

For  the  last  two  and  a  half  years,  I 
have  been  working  with  many  others  in 
the  Administration  to  improve  the 
capacity  of  information  systems  at 
federal,  state,  and  local  levels  to  meet 
these  needs,  and  to  see  how  we  can 


harness  emerging  electronic  data 
networks  to  support  population  health. 
A  key  point  that  has  emerged  from  this 
work  is  that,  as  our  need  for  health 
information  grows,  we,  at  the  federal 
level,  need  to  rethink  our  data 
collection  strategy. 

Simply  put,  current  approaches  are 
not  meeting  needs.   Externally,  our 
partners  in  states  and  communities  are 
urging  us  to  reduce  their  reporting 
burdens,  which  do  not  necessarily 
provide  them  with  useful  information. 
Internally,  it  is  becoming  increasingly 
apparent  that  our  currently  fragmented, 
poorly  coordinated,  and  episodic  data 
collection  efforts  have  resulted  in 
unnecessary  duplication,  critical  data 
gaps  (both  nationally  and  at  state  and 
local  levels) ,  competition  over  roles, 
and  inefficient  deployment  of  scarce 
information  resources. 

Fixing  the  problem  isn't  easy, 
however,  because  the  environment  in 
which  we  are  collecting  health-related 
information  is  changing  rapidly.   On  the 
one  hand,  the  health  system  itself  is  a 
moving  target,  both  in  the  way  personal 
health  care  services  are  delivered  and 
the  way  community-wide  public  health 
needs  are  being  met.   Dramatic  changes 
are  taking  place  in  the  organization  of 
each  of  these  sectors  of  the  health 
system,  and  in  the  way  they  interact 
(i.e.,  their  respective  roles, 
responsibilities,  relationships,  and 
funding  streams) .   Currently,  we  do  not 
even  have  a  typology  that  adequately 
characterizes  the  health  system  —  much 
less  one  that  still  will  be  meaningful 
five  or  ten  years  from  now. 

Another  complicating  factor  is  that 
advances  in  telecommunications  and  high 
performance  computing  are  changing  the 
way  people  communicate  with  one  another 
and  the  way  that  information  is 
recorded,  collected,  used.   Because  of 
the  efforts  of  many  in  this  room, 
considerable  headway  has  been  made  in 
the  planning  (and  now  early 
implementation)  of  logically  integrated 
health  information  systems.   These 
systems  take  many  different  forms,  but 
one  feature  they  have  in  common  is  that 
they  allow  information  collected 
routinely  in  the  course  of  delivering 
medical  care  or  public  health  services 
to  be  aggregated  and  used  for  multiple 
health-related  purposes. 

Integrated  information  systems  have 
the  potential  to  overcome  much  of  the 
duplication  and  fragmentation  that 
characterizes  current  data  collection 
efforts.   Yet  we  are  limited  in 
realizing  the  full  potential  of  these 
systems  because,  currently,  we  do  not 
have  a  national  framework  for  health 
information.   Such  a  framework  would 
include: 

-  uniform  data  standards  that  meet  the 
needs  of  the  different  types  of 


people  who  record  and  use  health 
information; 

-  a  uniform  system  of  unique 
identifiers  (not  just  for 
individuals,  but  also  for  other 
units  of  interest,  such  as 
providers,  worksites,  restaurants, 
wells,  and  employers) ; 

-  secure  environments  for  linking  data 
and  creating  anonymous  files; 

-  uniform  privacy  provisions  that 
protect  individuals  while  permitting 
critical  analytic  uses  of  health 
data ;  and 

-  appropriate  data  sharing  policies 
that  provide  communities  with  the 
complete  data  they  require  to  meet 
health  needs. 

A  third  element  in  the  changing 
environment  is  the  growing  movement 
toward  benchmarking  and  performance 
measurement.   Activities  at  various 
levels  —  including  Healthy  People  2000, 
health  plan  report  cards  (such  as 
HEDIS) ,  assessment  protocols  for  public 
health  agencies,  state  efforts  in 
benchmarking,  Performance  Partnership 
Grants,  and  the  measurement  of  federal 
program  performance  through  GPRA  and  the 
National  Performance  Review  —  are 
providing  a  substrate  for  clarifying 
goals  and  documenting  the  extent  to 
which  public  and  private  investments 
make  a  difference  in  population  health. 
Good  information  is  essential  to  the 
success  of  these  initiatives.   In 
particular,  we  need  to  develop  a  data 
infrastructure  that  supports  accurate, 
reliable,  and  comparable  measures  of 
results  that  can  be  achieved. 

Unfortunately,  however,  we  are 
living  in  an  environment  of  growing 
fiscal  constraints.   Consequently,  in 
spite  of  all  of  the  talk  about 
benchmarking  and  the  need  to  document 
value  for  dollars  spent,  budget 
pressures  and  the  movement  to  downsize 
government  are  making  it  increasingly 
difficult  to  obtain  adequate  funding  or 
personnel  for  the  data  activities  that 
support  these  measurements.   As  many  of 
you  are  aware,  appropriators  are  used  to 
thinking  about  data  as  an  administrative 
cost  rather  than  as  an  integral  part  of 
carrying  out  and  evaluating  health 
programs.   In  fact,  both  federal  and 
state  legislators  often  oppose  funding 
for  information  systems  because  they 
perceive  it  as  drawing  funds  away  from 
the  delivery  of  health  services.   This 
point  of  view  is  especially  ironic  in 
public  health,  where  data  collection  and 
analysis  undergirds  the  successful 
delivery  of  services.   In  a  constrained 
environment,  attention  to  information 
becomes  even  more  important  as  a  means 
of  targeting  scarce  dollars  toward 
services  that  can  accomplish  the  most  in 
terms  of  health. 

A  final  contextual  point  to  consider 
relates  to  partnerships.   For  various 


reasons,  the  current  environment  demands 
collaborative  relationships  between  the 
federal  government,  the  states,  and  the 
private  sector  in  data-related 
activities.   A  broad  range  of  expertise 
is  needed  to  put  health  information 
systems  into  place.   Data  to  protect  and 
improve  health  come  from  diverse  sources 
within  the  community.   Qualified  users 
span  many  different  professions.   And 
the  costs  of  information  systems  reguire 
a  broad  base  of  financing. 

Nonetheless,  several  factors  make  it 
difficult  to  move  forward  in  this 
regard.   On  the  one  hand,  there  is  a 
lack  of  consensus  about  the  appropriate 
role  of  the  federal  government  vis-a-vis 
the  states,  and  of  government  vis-a-vis 
the  private  sector.   Moreover,  many 
states  and  communities  lack  both  the 
policy  framework  and  the  structural 
mechanisms  to  bring  potential  partners 
in  health-oriented  information  systems 
together.   Without  clarity  about  roles, 
adeguate  incentives  to  collaborate,  or  a 
forum  for  social  interaction,  feasible 
data  strategies  to  support  population 
health  may  not  get  started  or  may  falter 
before  implementation. 

APPROACH  AND  ACTIVITIES  OF  THE  PHS  DATA 
POLICY  COMMITTEE 

A  year  and  a  half  ago,  the  Assistant 
Secretary  for  Health  established  a 
Public  Health  Service  (PHS) -wide  body  — 
the  Public  Health  Data  Policy 
Coordinating  Committee  —  to  highlight 
the  guintessential  role  of  information 
in  public  health  and  to  develop, 
coordinate,  and  implement  data  policies 
in  support  of  population  health.   That 
committee  has  highlighted  the  importance 
of  adopting  a  broad  systems  approach  to 
health  information  in  the  future, 
involving: 

-  data  from  integrated  health 
information  systems  as  well  as 
surveys ; 

-  a  balanced  approach  to  data 
collection,  including  information 
about:  (1)  health  and  functional 
status,  (2)  behavioral, 
environmental,  occupational,  and 
infectious  risks  to  health,  (3)  the 
capacity  and  functioning  of  the 
medical  and  public  health  systems; 
and  (4)  the  costs,  utilization,  and 
financing  of  individual  and 
population-based  health  services; 

-  the  capacity  to  link  different  types 
of  data  together  and  to  aggregate 
them  geographically  so  that  it  is 
possible  to  look  at  important 
interrelationships  (for  example, 
between  health  status  and  medical 
care;  personal  risk  behaviors  and 
sociodemographics ;  environmental 
hazards  and  health  care  costs;  and 
the  emergence  of  managed  care  and 
funding  for  essential  public  health 
services) ; 


-  comparable  health-related 
information  at  national,  state,  and 
local  levels; 

-  meaningful  and  reliable  data  to 
support  program  measurement  and 
performance  measurement;  and 

-  federal/state  and  public/private 
partnerships  to  develop  data 
strategies  and  to  collect 
information. 

In  its  mission  to  improve  data  in 
support  of  population  health,  the 
Committee  has  focused  on  seven  policy 
areas:  the  National  Information 
Infrastructure  (Nil) ;  health  data 
standards;  privacy;  integrated 
information  systems;  performance 
measurement;  survey  strategy;  and  health 
statistics.   Its  activities  and  progress 
in  these  areas,  which  I  will  outline  in 
the  remainder  of  my  talk,  are  the 
product  of  a  close  working  relationship 
with  a  broad  range  of  players  involved 
in  health  and  information:  public  health 
professionals  at  state  and  local  levels; 
other  relevant  components  of  the  federal 
government;  clinicians;  researchers; 
experts  in  informatics,  high  performance 
computing,  and  telecommunications;  and 
states  and  communities  that  have  been 
moving  forward  with  integrated 
information  systems. 

The  Nil  and  Population-Based  Public 
Health 

The  purpose  of  the  National 
Information  Infrastructure  Initiative  is 
to  enhance  the  basic  infrastructure  for 
telecommunications  and  computer 
technology  in  all  sectors  of  the  U.S. 
economy.   From  the  start,  health  has 
been  identified  as  one  of  the  key 
sectors  that  can  benefit  from  Nil 
technology.   But  so  far,  Nil  grants 
related  to  health  have  primarily 
supported  applications  of  high 
performance  computing  and  telemedicine 
to  the  delivery  of  medical  care  to 
individuals.   Relatively  little 
attention  has  been  paid,  by  either  the 
private  or  the  public  sector,  to 
applications  that  could  improve  the 
capacity  of  communities  to  carry  out  the 
nonclinical  or  population-based 
functions  of  public  health  (i.e., 
services  that  identify  local  health 
problems,  prevent  epidemics  and  the 
spread  of  disease,  protect  against 
environmental  hazards,  and  assure  the 
guality  and  accessibility  of  health 
services) . 

Last  April,  the  PHS  sponsored  a 
conference  at  the  National  Library  of 
Medicine,  during  which  leaders  in  the 
Nil  and  population  health  communities 
had  an  opportunity  to  come  together,  to 
explain  their  work  to  each  other,  to 
delineate  the  barriers  that  currently 
discourage  application  of  Nil 
technologies  to  the  information  problems 
of  population-based  public  health,  and 


to  lay  out  a  comprehensive  strategy  for 
moving  forward.   At  this  conference, 
five  barriers  emerged,  above  and  beyond 
basic  resource  constraints  and  the 
limited  appreciation  by  both  the  public 
and  policymakers  of  the  importance  of 
population-based  public  health.   These 
include: 

-  lack  of  nationally  uniform  policies 
to  protect  privacy  while  permitting 
critical  analytic  uses  of  health 
data; 

-  a  lack  of  nationally  uniform,  multi- 
purpose data  standards  that  meet  the 
needs  of  the  diverse  groups  who 
record  and  use  health  information; 

-  insufficient  awareness  of  the 
applicability  of  Nil  technologies  in 
meeting  the  information  needs  of 
population-based  public  health; 

-  a  public  health  workforce  that  lacks 
essential  information  technology 
skills;  and 

-  organizational  and  financing  issues 
that  make  it  difficult  to  integrate 
information  systems  or  bring 
potential  partners  together. 

Those  attending  a  strategy  session 
following  the  conference  proposed  a 
strategic  plan  that  capitalizes  on  what 
a  broad  range  of  actors  —  state  and 
local  public  health  agencies,  federal 
agencies,  professional  associations, 
educational  institutions,  and  other 
groups  —  can  do  individually  and 
together  to  overcome  these  barriers.   A 
copy  of  the  white  paper  was  included  in 
your  conference  materials.   If  you  would 
like  to  learn  more  about  this  project  or 
get  involved  in  the  strategic  plan, 
please  contact  Betsy  Humphreys  at  the 
National  Library  of  Medicine  or  any  of 
the  other  people  listed  at  the  end  of 
the  white  paper. 

Health  Data  Standards 

Both  the  PHS  and  the  Department  of 
Health  and  Human  Services  have  begun 
work  on  federal  aspects  of  the  white 
paper  strategic  plan.   Following  a 
mandate  from  the  Vice  President,  the 
Department  is  establishing  a  Department- 
wide  Data  Council  to  develop  and 
coordinate  policy  in  the  areas  of  data 
standards,  privacy,  and  data  collection. 
The  Council  will  interact  with  other 
Departments  and  Agencies  in  the 
Administration,  and  with  an  external 
advisory  committee  representing  the 
views  of  the  broad  range  of 
stakeholders.   A  key  focus  of  the  PHS 
Data  Policy  Committee  in  this  regard  has 
been  to  emphasize  the  need  for  the 
Department  to  work  closely  with  the  full 
range  of  parties  in  the  public  and 
private  sectors  who  collect  and  use 
health-related  information  and  to 
support  uses  of  information  that  promote 
population  health  as  well  as  cost 
containment  and  medical  care. 


In  the  area  of  health  data 
standards,  there  is  a  growing  awareness 
in  both  PHS  and  the  broader  Department 
that  the  key  to  this  issue  is  not  simply 
reaching  any  agreement  (for  example,  by 
locking  stakeholders  in  a  room  until 
they  pick  one  of  many  coding  systems  to 
use) .   The  real  challenge  is  to  identify 
and  implement  standards  that  meet  the 
needs  of  the  diverse  groups  that  use 
health  information.   Without 
multipurpose  standards,  data  recorded  in 
electronic  information  systems  may  not 
be  accurate.   If  information  is  entered 
in  these  systems  in  a  nomenclature  that 
is  neither  meaningful  to  nor  actively 
used  by  the  people  who  are  recording  it, 
those  people  are  unlikely  to  have  much 
interest  in  whether  the  information  they 
record  is  accurate  or  not.   And  if 
information  that  is  recorded  for  one 
purpose  cannot  also  be  used  for  other 
purposes,  we  are  not  going  to  achieve 
the  administrative  simplification  that 
so  many  are  talking  about. 

In  the  current  environment,  which 
some  have  likened  to  a  "Tower  of  Babel," 
it  is  very  hard  to  move  forward  with 
integrated  information  systems.   States 
attempting  to  do  so  must  develop  their 
own  standards,  negotiating  with  major 
health  care  payers  and  providers  as  well 
as  other  parties  whose  data  are  critical 
to  meeting  health  information  needs. 
These  efforts  are  not  only  time- 
consuming,  they  risk  the  adoption  of 
standards  that  are  more  suitable  for  one 
purpose,  such  as  paying  claims,  than  for 
meeting  the  needs  of  clinicians, 
researchers,  and  the  public  health 
community.   When  standards  differ  over 
geographic  boundaries  and  data  systems, 
it  is  impossible  to  aggregate 
information. 

Technically,  there  may  be  a  solution 
to  the  standards  problem  in  the  near 
future.   The  Unified  Medical  Language 
System  (UMLS) ,  which  is  being  developed 
under  the  auspices  of  the  National 
Library  of  Medicine,  is  a  vocabulary 
that  was  originally  developed  to 
retrieve  medical  information  from  the 
literature.   Currently,  it  is  also  being 
used  as  the  basis  for  computerized 
medical  records.   With  over  a  quarter 
million  concepts  currently  defined,  it 
will  soon  be  possible  to  record  data  in 
integrated  information  systems  in  the 
same  detailed  and  meaningful  form  in 
which  they  are  entered  in  computerized 
patient  records  (i.e.,  a  form  that  works 
for  clinicians  when  they  take  care  of 
patients) . 

The  real  potential  of  the  UMLS, 
however, » stems  from  the  fact  that 
information  recorded  in  this  vocabulary 
can  be  mapped  to  any  of  a  broad  range  of 
coding  and  classification  systems,  all 
of  which  are  contained  in  the  UMLS 
metathesaurus.   That  means  that 
information  can  be  recorded  in  an 
unbiased  form,  useful  for  patient  care 


at  the  point  it  is  being  delivered,  and 
then  can  be  translated  automatically 
into  coding  systems  appropriate  for 
other  purposes,  such  as  CPT  for  billing 
or  ICD  for  morbidity  or  mortality 
reporting. 

Until  recently,  the  UMLS  has  focused 
primarily  on  medical  treatment. 
Currently,  however,  the  National  Library 
of  Medicine  is  working  to  expand  the 
UMLS  to  unique  public  health  concepts, 
making  it  a  true  health  vocabulary  that 
can  accept  information  from  machine 
readable  public  health  records  as  well 
as  clinical  records.  In  addition,  the 
National  Library  of  Medicine  is 
beginning  to  fund  demonstrations  to  test 
the  UMLS  in  clinical  and  public  health 
settings  as  well  as  in  integrated  health 
information  systems.   Systems  designed 
with  the  flexibility  to  accept 
information  in  this  form  will  be  able  to 
capitalize  on  the  increasing  use  of  the 
UMLS  by  clinicians  and  public  health 
professionals  in  their  computerized 
records. 

Privacy  of  Health  Information 

The  PHS  Data  Policy  Committee  has 
addressed  the  issue  of  privacy  from  the 
framework  of  population  health.   In  this 
context,  protecting  the  privacy  of 
individually-identifiable  information  is 
critical  since  inappropriate  use  of 
health  information  can  harm  people  and 
be  a  health  hazard  in  itself.   But 
supporting  the  usefulness  of  health  data 
is  equally  important  since,  without 
information,  it  is  simply  not  possible 
to  improve  health.   Consequently,  the 
challenge  is  to  find  ways  to  protect  the 
privacy  of  individuals  while  permitting 
critical  analytic  uses  of  data. 

Although  this  is  a  difficult  task, 
it  may  not  be  impossible  to  achieve, 
especially  if  we  keep  in  mind  that  many 
uses  of  data  to  support  population 
health  do  not  require  the  user  to 
identify  any  particular  individual. 
Identifiers  are  required  in  order  to 
link  data  and  create  useful  analytic 
files.   Thereafter,  files  can  be 
stripped  of  identifiable  information  and 
still  be  of  benefit  for  many  (although 
certainly  not  all)  health-related 
purposes.   Viewed  from  this  perspective, 
policies  to  support  privacy  and  the 
usefulness  of  health  information  need  to 
incorporate  both  protections  for 
individually-identifiable  health 
information  and  secure  environments  to 
link  identifiable  data  and  to  create 
anonymous,  public  use  files. 

Currently,  without  these  types  of 
policies  in  place,  some  state  and 
federal  regulations  protect  privacy  by 
preventing  the  linkage  of  data  from 
different  systems,  seriously  hampering 
the  identification  of  health  problems  or 
the  ability  to  take  effective  action. 
In  states  without  sufficient  or  clear 
privacy  protections,  providers  and 


consumers  may  be  reluctant  to  provide 
identifiable  information  for  statewide 
databases.   These  barriers  impede  our 
ability  to  address  interstate  health 
problems,  which  affect  half  of  all 
Americans  who  live  near  a  state  border. 

Interestingly,  quite  a  bit  of 
progress  in  this  area  was  made  in  the 
Congress  last  year  in  the  context  of 
health  care  reform.   Both  Representative 
Condit  and  Senator  Leahy  incorporated 
comprehensive  privacy  provisions  in 
information  systems  bills.   These  bills, 
which  had  bipartisan  support  in  both 
houses  of  the  Congress,  balanced  privacy 
and  access  to  data  by  incorporating  not 
only  provisions  to  protect  the  privacy 
of  individually-identifiable  health 
information  (in  all  forms),  but  also 
secure  environments  for  linking  data  and 
creating  anonymous  files. 

If  passed,  these  bills  would  have 
established  a  national  framework  for 
protecting  privacy  while  assuring  access 
to  health  information  to  promote  the 
public  good.   But  it  is  important  to 
keep  in  mind  that  these  bills  were 
prepared  in  the  context  of  health  care 
reform  —  a  reform  in  which  there  were 
rules  of  the  game  requiring  health  plans 
to  provide  information  and  preventing 
them  from  using  health  information  for 
such  nefarious  purposes  as  redlining  and 
cherrypicking.   In  the  current 
unregulated  environment,  a  pro-data  and 
pro-privacy  strategy  is  far  more 
difficult  to  achieve  because  even  public 
use  files  have  the  potential  to  harm 
people. 

Currently,  many  groups  are  involved 
in  efforts  to  educate  policymakers  about 
the  chilling  effect  of  the  lack  of 
privacy  legislation  on  the  development 
of  integrated  health  information 
systems.   The  Departmental  Data  Council 
is  planning  to  begin  its  activities  in 
this  area  by  working  on  the  development 
of  model  laws,  and  by  identifying  best 
practices  for  protecting  the  privacy  of 
health  information  and  for  ensuring 
security  in  electronic  data  systems. 

Integrated  Health  Information  Systems 

An  area  in  which  the  PHS  Data  Policy 
Committee  has  been  particularly  active 
involves  support  for  the  development  of 
integrated  health  information  systems  at 
the  state  level.   Numerous  projects  are 
currently  underway.   Some  will 
facilitate  electronic  claims  processing 
and  coverage  verification  in  the  health 
care  system.   Others  are  designed  to 
integrate  the  delivery  of  health  care  to 
individuals  across  numerous  state  and 
local  agencies.   In  broader  public 
health-oriented  systems,  clients 
encompass  not  only  people,  but  also 
restaurants,  hospitals,  waste  systems, 
swimming  pools,  and  wells,  and  the 
Internet  is  used  to  link  public  health 
agencies  to  each  other  and  to  academic 
health  centers. 


Although  these  developmental  efforts 
are  somewhat  different  in  approach,  they 
raise  common  policy,  implementation,  and 
political  issues  with  which  many  in  the 
public  and  private  sector  are  grappling. 
At  the  present  time,  however,  project 
developers  have  little  opportunity  to 
benefit  from  the  knowledge  or 
experiences  of  others.   States  and  the 
federal  government  have  not  established 
a  mechanism  for  assessing  the  impact  of 
proposed  changes  in  their  information 
systems  on  each  other.   Moreover,  the 
categorical  nature  of  many  health 
programs  impedes  the  development  of 
integrated  information  systems,  both 
through  categorical  information  systems 
that  speak  different  languages  (in  terms 
of  vocabulary,  software,  and  data 
standards)  and  through  prohibitions  on 
the  use  of  categorical  funds  for 
developing  or  maintaining  information 
systems  that  benefit  programs  in 
addition  to  the  one  for  which  the  funds 
were  appropriated. 

The  PHS  Data  Policy  Committee  has 
held  several  meetings  with  leading  edge 
states  to  identify  common  concerns  and 
to  develop  a  productive  course  of 
action.   Future  meetings  will  bring 
together  representatives  from  states  and 
relevant  federal  agencies  to  discuss 
such  issues  as  the  UMLS,  privacy,  and 
the  impact  of  proposed  changes  in 
Departmental  data  systems  on  state 
integrated  information  systems. 

Another  project  just  getting 
underway  is  the  construction  of  a 
registry  of  state  developments  in 
integrated  health  information  systems. 
This  registry  is  intended  to  serve  as 
more  than  a  compendium  of  projects.   It 
is  envisioned  as  a  dynamic  database  of 
current  information  that  will  be  updated 
regularly  by  system  developers  and  that 
can  be  accessed  and  searched  easily  by  a 
broad  audience  of  users.   To  achieve 
this  goal,  the  content,  format,  and 
indexing  of  the  database  for  electronic 
searching  will  be  developed  in  close 
collaboration  with  those  who  will  be 
providing  information  to  the  system  as 
well  as  those  who  will  be  using  it. 
Various  mechanisms  will  be  tested  for 
dissemination,  including  a  World  Wide 
Web  Server. 

Through  the  newly  proposed 
Performance  Partnership  Grants  (PPGs) , 
which  I  will  discuss  in  more  detail 
later  in  this  conference,  the  Committee 
put  forward  policies  that,  if  enacted, 
will  substantially  reduce  the  number  of 
categorical  information  systems  while 
giving  states  the  flexibility  to  use  PHS 
grant  funds  to  develop  and  maintain 
integrated  information  systems  that 
benefit  multiple  programs.   Importantly, 
the  new  grants  would  protect  current 
levels  of  funding  for  surveillance,  and 
would  exclude  funds  related  to  data 
collection  and  analysis  from  the  state 
administrative  cap. 


Public  Health  Performance  Measurement 

Nothing  highlights  the  importance  of 
data  more  than  performance  measurement, 
and  the  PHS  Data  Policy  Committee  has 
been  working  through  two  mechanisms  to 
advance  the  data  infrastructure  for 
performance  measurement  in  public 
health:  Performance  Partnership  Grants 
and  an  Institute  of  Medicine  (IOM)  panel 
on  the  use  of  performance  measurement  to 
improve  community  health. 

Performance  Partnership  Grants 
establish  a  new  basis  for  grant 
relationships  between  the  federal 
government  and  the  states,  in  which  the 
two  will  work  together  as  partners  to 
achieve  measurable  health  objectives. 
Considerable  groundwork  in  this  area  has 
already  been  set  through  the  Healthy 
People  2000  process,  through 
benchmarking  efforts  in  a  number  of 
states,  and  through  federal  activities 
such  as  GPRA  and  the  National 
Performance  Review.   An  internal 
feasibility  study  conducted  by  the  PHS 
earlier  this  provided  convincing 
evidence  that,  although  data  collection 
efforts  can  clearly  be  improved, 
meaningful  public  health  results  can  be 
measured  now  through  existing  federal- 
state  data  systems. 

This  fall,  the  Department  of  Health 
and  Human  Services  will  initiate  a 
partnership  process  to  identify  and 
measure  objectives  for  the  PPGs.   The 
process  will  consist  of  a  series  of 
regional  meetings  to  obtain  structured 
input  from  a  broad  range  of  stakeholders 
—  states;  local  governments;  tribal 
governments;  public  health 
professionals;  other  public,  private, 
and  professional  groups;  interested 
citizens;  and  federal  officials  —  about 
the  results  they  want  to  achieve,  and 
believe  it  is  feasible  to  achieve, 
through  these  programs. 

Working  from  information  generated 
through  the  regional  meetings,  an 
independent  technical  panel  with 
comprehensive  knowledge  of  national  and 
state  data  systems  will  identify  which 
of  these  results  can  be  measured  now  as 
objectives,  and  recommend  how  federal- 
state  data  systems  could  be  improved  to 
measure  other  important  PPG  results  in 
the  future.   The  panel's  report  will  be 
used  by  the  Department  to  prepare  a 
provisional  menu  of  objectives  for  each 
PPG,  which  will  be  distributed  widely 
for  review  and  comment  before  it  is 
finalized.   As  changes  in  federal  and 
state  data  systems  are  implemented  in 
the  future,  the  menu  of  PPG  objectives 
will  be  periodically  refined  to  more 
closely  reflect  the  results  that  states, 
the  federal  government,  and  interest 
groups  want  to  achieve. 

While  the  regional  meetings  and 
technical  panel  are  designed  to  support 
implementation  of  PPGs,  their  importance 
extends  far  beyond  this  particular 
legislative  initiative.   The  goal  of  the 


process  is  to  lay  a  strong  foundation 
for  a  continuing  federal-state 
partnership  in  data  collection  and 
benchmarking.   This  should  enhance  our 
ability  to  document  how  federal  and 
state  investments  make  a  difference  in 
public  health,  regardless  of  the  way 
federal  grant  programs  are  structured. 

After  states  or  communities  identify 
the  improvements  in  health  outcomes  they 
would  like  to  achieve,  they  still  need 
to  identify  the  range  of  actors  that  can 
make  an  impact  on  these  results,  clarify 
the  specific  roles  various  actors  can 
play,  monitor  the  extent  to  which  these 
actions  make  a  difference,  and  promote  a 
collaborative  environment  in  which 
providers  of  personal  health  care 
services,  essential  public  health 
services,  and  other  groups  can  work 
constructively  together  toward  common 
goals.   The  IOM  Committee  on  Using 
Performance  Monitoring  to  Improve 
Community  Health  has  been  charged  with 
looking  at  the  potential  usefulness  of  a 
comprehensive  system  of  public  health 
performance  monitoring  in  addressing 
these  issues,  in  effect,  by  thinking 
about  such  a  system  as  a  community-wide 
health  management  tool.   In  its  report, 
the  IOM  Committee  will  illustrate  such  a 
process  by  identifying  sets  of 
indicators  that  can  measure  the 
performance  of  a  broad  range  of  actors 
in  addressing  specific  health  problems, 
and  by  generating  a  "tool-box"  that  can 
help  others  develop  sets  of  indicators 
for  their  high  priority  health  problems. 

Survey  Strategy 

Many  of  you  are  aware  of  the 
Department's  plans  to  rethink  its  data 
collection  strategy,  in  particular,  its 
newly  proposed  survey  integration  plan, 
in  which  the  National  Health  Interview 
Survey  would  function  as  a  sampling 
"nucleus"  for  many  population  surveys, 
and  as  the  hub  for  a  series  of 
longitudinal  panels,  including  one 
providing  better  information  on  medical 
insurance  and  expenditures.   As  these 
efforts  move  forward,  the  PHS  Data 
Policy  Committee  has  emphasized  the  need 
to  have  broad  and  structured  input  into 
the  decisionmaking  process  so  that  the 
Department's  data  collection  strategy  is 
rationalized  through  a  framework  that 
supports  population  health.   If  all 
stakeholders  have  a  voice  at  the  table, 
we  will  be  in  a  better  position  to 
ensure  that  strategies  to  expand  one 
type  of  data  (for  example,  expenditures) 
do  not  come  at  the  expense  of  critical 
information  related  to  health  status  and 
risks  to  health.   Equally  important,  we 
will  begin  building  a  constituency  that 
can  better  educate  appropriators  about 
the  importance  of  data  collection 
activities  in  improving  population 
health. 

With  respect  to  specific  projects, 
the  PHS  Data  Policy  Committee 


incorporated  a  $4  million  initiative  in 
the  President's  1996  budget  request  to 
obtain  comprehensive  data  on  the 
capacity  and  functioning  of  the  public 
health  infrastructure  (i.e.,  the  full 
range  of  federal,  state  and  local 
government  agencies  responsible  for  the 
health  of  the  population) .   Proposals 
are  currently  being  solicited  to  develop 
a  strategy  for  collecting  these  data. 
The  contractor  will  work  through  an 
intergovernmental  partnership,  building 
on  data  currently  being  collected  and  on 
work  that  has  already  been  accomplished 
in  developing  consensus  definitions  for 
essential  public  health  services, 
evaluating  public  health  performance, 
and  obtaining  information  on  public 
health  expenditures.   The  project  is 
intended  to  bring  into  a  cohesive 
framework  many  disparate  data  collection 
efforts.   By  doing  so,  it  will  provide 
policymakers  at  many  levels  with 
information  to  assess  the  impact  of 
rapid  changes  in  the  health  care  system 
on  the  public  health  infrastructure,  and 
will  serve  as  a  comprehensive  basis  for 
health  services  research  in  public 
health. 

Other  components  of  the  President's 
1996  budget  request  would  address  data 
gaps  and  improve  analytic  capacity  in 
various  ways:  by  expanding  available 
information  on  disabled  persons  and 
health  providers  (including  substance 
abuse  and  mental  health  providers) ,  by 
improving  the  National  Health  Accounts, 
by  strengthening  the  Area  Resource  File, 
and  by  enhancing  the  linkage  of 
information  related  to  population 
characteristics,  health  problems,  and 
health  resources  at  the  community  level. 
As  part  of  its  survey  rationalization 
process,  the  Department  is  planning  to 
work  on  several  other  fronts  as  well: 
facilitating  federal/state  partnerships 
in  health  surveys,  consolidating 
provider  surveys,  improving  the 
usefulness  of  its  capacity  surveys  by 
developing  a  workable  typology  to 
characterize  the  evolving  health  system, 
and  integrating  its  health  surveillance 
and  grant  reporting  systems. 

Health  Statistics 

As  part  of  the  Administration's 
reinventing  government  initiative,  the 
Department  has  given  priority  to 
strengthening  its  capabilities  in  health 
statistics.   Moving  in  this  direction  is 
important  not  only  to  improve  the 
availability  of  information  for  setting 
the  nation's  health  agenda,  but  also  to 
undergird  health  policy  decisions  with 
sound,  credible  data.   Since  health 
statistics  are,  in  effect,  a  "public 
utility"  covering  a  broad  range  of 
issues  —  including  health  status,  risks 
to  health,  the  delivery  of  individual 
and  community-wide  health  services,  the 
capacity  of  the  health  care  and  public 
health  systems,  and  health  expenditures 


—  the  benefits  of  strengthening  this 
function  extend  far  beyond  the 
Department  to  other  agencies  in  the 
federal  government,  to  policymakers  and 
health  professionals  at  state  and  local 
levels,  to  the  private  sector,  to 
advocacy  groups,  and  to  various  segments 
of  the  research  community. 

As  options  are  being  considered,  the 
PHS  Data  Policy  Committee  has 
highlighted  a  number  of  points.   The 
essence  of  statistics,  turning  raw 
numbers  into  useful  information, 
reguires  a  comprehensive  view  of  health 
data  and  a  close  working  relationship 
with  a  broad  range  of  users.   At  the 
same  time,  however,  it  is  critical  that 
the  statistics  function  remains  outside 
of  the  political  or  policy  fray  —  in 
other  words,  that  its  operations  and 
analyses  remain  impartial  and  unbiased. 
A  strong  statistics  agency,  therefore, 
needs  to  be  viewed  as  independent,  and 
avoid  the  appearance  or  reality  of  being 
captured  by  any  given  user  or  point  of 
view. 

As  recent  events  make  clear,  there 
is  also  a  need  for  a  statistics  agency 
that  can  function  as  a  "center  of 
gravity"  around  which  a  vision  for 
useful,  coordinated  data  systems  can  be 
formed.   Such  an  agency  could 
institutionalize  the  progress  that  has 
been  made  thus  far  in  integrating 
general  purpose  health  surveys,  bringing 
some  but  not  all  of  these  surveys  under 
one  roof.   It  could  facilitate  the 
establishment  of  information 
partnerships  with  states  and  the  private 
sector,  coordinating  health  surveys 
conducted  at  multiple  levels,  and 
creating  opportunities  for  greater  use 
of  data  from  administrative  and 
programmatic  systems.   It  could  function 
as  a  methodologic  resource  center, 
providing  the  Department  and  others  with 
a  critical  mass  of  expertise  to  support 
statistical  analyses  and  to  develop  the 
indicators  that  undergird  performance 
measurement.   In  addition,  it  could 
serve  as  a  steady  voice  for  a  broad- 
based,  balanced  program  of  data 
collection  committed  to  meeting  the 
needs  of  a  wide  range  of  data  users. 

Consideration  of  these  issues  within 
the  Department  comes  at  an  opportune 
time,  as  the  search  process  has  started 
for  a  new  Director  for  the  National 
Center  for  Health  Statistics,  as  major 
changes  are  being  made  in  the 
Department's  data  policy  function  and 
data  collection  strategy,  and  as  the 
Congress  considers  steps  to  more  closely 
coordinate  or  combine  federal  statistics 
agencies.   It  also  comes  at  a  critical 
time.   Unless  we  are  successful  in 
identifying  what  really  needs  to  be 
measured  and  in  forming  partnerships  to 
collect  this  information,  the  nation 
will  not  be  able  to  afford  to  meet  its 
health  data  needs.   And  unless  we  can 
demonstrate  to  policymakers  and  the 


general  public  the  usefulness  and 
practical  value  of  health  data  to  them, 
appropriators  will  be  unlikely  to 
provide  the  federal  government  or  the 
states  with  adeguate  data  funding. 
According  to  a  story  I  heard  recently, 
which  I  hope  is  not  apocryphal,  the  head 
of  health  statistics  in  a  nearby  country 
regularly  attends  Cabinet  meetings, 
providing  information  on-line  and  in 
user-friendly  graphic  displays  for 
federal  policymakers.   I  can't  think  of 
a  better  way  to  highlight  the 
guintessential  role  of  information  in 
population  health. 

Various  organizational  and 
functional  options  for  a  statistics 
agency  are  now  being  considered,  both  in 
the  Department  and  the  Congress.   In 
this  matter,  and  for  many  other  data 
issues  that  I  covered  in  my  talk,  now  is 
the  time  for  those  actively  involved  in 
health  data  collection  and  analysis  to 
keep  abreast  of  developments  and  to 
bring  their  perspectives  to  bear  in  the 
policymaking  process.   Broad,  balanced, 
and  active  input  is  the  sine  gua  non  of 
informed  decisionmaking,  especially  in 
times  of  rapid  change.   Your  input  has 
been  important  thus  far  and  will  be  even 
more  important  in  the  future.   We,  in 
the  Public  Health  Service  and  the 
Department  of  Health  and  Human  Services, 
look  forward  to  working  with  you  on 
these  vital  issues. 
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The  ability  to  assess  the  health  of  the 
public  in  a  society  or  any  political  subdivision 
of  a  society  is  a  function  of  the  quality  and 
detail  of  the  data  that  is  available. 
Demographic  data  in  the  form  of  births,  deaths 
and  age  distribution  in  the  population 
constitute  a  minimum  that  may  not  even  be 
available  in  some  societies  lacking  the 
infrastructure  to  gather  such  data.  In  countries 
such  as  the  former  Soviet  Union  where  such  data 
were  available,  the  measurable  derivatives  such 
as  life  expectancy  at  birth  or  at  any  age,  and 
more  specifically  the  infant  mortality  rate, 
showed  a  deterioration  of  the  state  of  the 
health  of  the  public  that  preceded  the  breakdown 
of  that  particular  political  and  economic 
system. 

Within  the  United  States  the  National 
Center  for  Health  Statistics  since  its  inception 
in  1962   has  been  charged  with  the  surveillance 
of  the  health  of  the  U.S.  public.   In  a  nation 
that  is  concerned  with  the  right  to  privacy  of 
its  citizens  to  a  much  higher  degree  than  many 
other  nations  at  a  similar  level  of  economic 
development  NCHS  has  provided  us  with  a 
continuing  record  of  measures  of  the  health  of 
the  U.S.  public  that  are  the  envy  of  other 
nations . 

The  efforts  of  the  National  Center  for 
Health  Statistics  for  the  U.S.  have  provided  us 
with  an  exemplary  record  of  tracking  the  state 
of  the  health  of  the  nation  over  a  nearly  forty 
year  period  with  a  degree  of  sensitivity  and 
specificity  that  can  inspire  pride  of 
achievement  while  at  the  same  time  providing  us 
with  a  keen  awareness  of  challenges  that  are  not 
being  met.   International  comparisons  of  life 
expectancy  and  infant  mortality  in  countries 
that  are  at  a  high  level  of  economic  development 
generally  show  that  there  is  a  price  to  pay  for 
the  amount  of  heterogeneity  in  education, 
socialization,  employment  opportunity  and  income 
that  a  society  is  experiencing.   As  a  result, 
comparisons  between  Sweden  or  Switzerland  on  the 
one  hand,  with  limited  diversity,  and  the  United 
States,  with  considerable  diversity,  tend  to 
rate  the  health  of  the  public  better  in  Sweden 
and  Switzerland  than  in  the  U.S. 

In  public  health  the  twin  objectives  of 
the  maintenance  of  health  and  the  prevention  of 
disease  are  primary.   NCHS  maintains  a 
continuing  historical  record  of  the  health  of 
the  public  measured  in  a  variety  of  ways, 
especially  tracking  agreed  upon  national 
objectives  such  as  the  Health  Objectives  for 
1990  and  for  the  Year  2000.   There  will  likely 
always  be  a  tension  between  the  objective  of 
preventing  disease  and  premature  death  and  the 
objective  of  curing  disease  and  postponing 
death.   This  tension  manifests  itself  in  terms 
of  resource  allocation  and  organization  of 
health  services.  A  good  example  is  collecting 
data  on  childhood  immunization  which  clearly 
should  have  a  high  priority,  but  its  benefits 
are  delayed,  its  costs  are  immediate,  and 
tracking  is  not  simple.   In  addition  there  are 
difficulties  in  the  way  the  public  health 
systems  and  the  curative  health  care  systems  are 
organized . 

Loss  of  access  to  medical  care  and  the 
rate  of  rise  in  the  cost  of  medical  care  have 
become  important  national  concerns  and  unless 
suitably  addressed  are  likely  to  express 
themselves  as  eventual  reductions  in  the  state 
of  health  of  our  population. 

Decision  makers  in  the  public  as  well  as 
in  the  private  sector  are  facing  complex 
questions,  and  ongoing  and  anticipated 
transitions  in  the  health  care  system.   In  the 
last  year  I  have  been  receiving  an  unusual 


number  of  requests  by  Health  Maintenance 
Organizations  and  insurance  companies  to  provide 
them  with  the  names  of  recent  doctoral  graduates 
who  could  help  them  start  programs  in  the  area 
of  outcome  research.   Judging  by  the  beginning 
salaries  they  are  offering,  they  are  placing  a 
high  priority  on  these  new  efforts. 

The  Federal  Government  is  also  in  the 
process  of  increasing  the  amount  of  information 
collected  in  various  surveys  that  would  help  in 
the  description  and  analysis  of  different  forms 
of  organization  of  health  care  in  terms  of 
access,  cost,  acceptance  and  outcome.   Part  of 
the  motivation  for  this  process  is  driven  by  the 
wish  to  "reinvent  government",  and  another 
motivation  is  to  increase  the  amount  of  data 
collection  and  analysis  relevant  to  health  care 
policy. 

As  a  frequent  user  of  data  collected  by 
various  Government  agencies  and  as  one  who  has 
collected  population  based  data  for  special 
studies  on  a  number  of  occasions,  I  am  somewhat 
concerned  when  important  surveys  such  as  the 
Health  Interview  Survey  are  being  redesigned  in 
their  basic  structure  in  order  to  accomodate 
current  health  care  policy  needs. 

Existing  surveys  with  a  long  and 
successful  history  have  arrived  at  the  current 
form  and  format  as  the  result  of  continuing 
reviews  and  negotiation  with  a  number  of  stake 
holder  organizations  within  and  outside  the 
Government.   Many  advocacy  groups  have  to  rely 
on  the  results  of  such  surveys  to  monitor  the 
comparative  state  of  health  of  their 
constituencies  as  well  as  any  trends  that 
develop  over  time.   Reductions  in  the  coverage 
might  threaten  the  continuity  over  time  of  such 
surveillance. 

Population  based  surveys  and  the 
instruments  on  which  they  rely  are  the  product 
of  numerous  compromises  in  conflicting  factors 
such  as : 

■  the  amount  of  information  obtained 

■  the  burden  on  the  respondent 

■  the  response  rate 

■  the  quality  of  the  information  obtained 

■  the  cost  of  the  survey 

We  can  assume  that  existing  surveys  have 
been  optimized  with  regard  to  such  factors  as 
listed  above.  The  advent  of  new  technology  in 
interviewing  such  as  Computer  Assisted  Personal 
Interviews  and  Computer  Assisted  Telephone 
Interviews  brings  about  perturbations  in 
currently  optimized  surveys. 

When  a  large  number  of  surveys  are  in  the 
field  it  is  always  useful  to  consider  whether 
there  are  overlaps  in  data  collected,  whether  a 
particular  sample  could  serve  more  than  one 
survey,  or  whether  development  costs  could  be 
reduced  by  consolidation.   Tempting  as  such 
proposals  for  consolidation  are,  it  is  extremely 
important  to  scrutinize  the  savings  claimed  for 
such  consolidation  efforts.   Each  survey  is 
already  the  result  of  an  optimization  of  a 
number  of  conflicting  demands  of  the  type  listed 
above.   Consolidation  of  one  or  more  surveys 
will  likely  result  in  a  new  optimization  in 
which  one  or  more  of  the  existing  surveys  will 
be  diminished  in  favor  of  new  objectives. 

As  an  example  one  or  more  of  the  following 
is  likely  to  occur: 

■  if  costs  are  to  be  lowered  then  the  product 
of  sample  size  and  the  amount  of  data 
collected  for  all  surveys  under  consideration 
will  be  likely  to  be  reduced 

■  if  two  or  more  surveys  are  merged,  the 
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individual  respondent  burden  can  not  be 
increased  without  reducing  participation 
rate,-  it  is  therefore  likely  that  each 
consolidated  survey  will  have  to  give  up  a 
substantial  fraction  of  its  content 

■   although  additional  joint  distributions  will 
be  recorded,  others  will  be  lost,  and  the 
result  will  be  a  shift  of  resources  into  the 
new  joint  distributions,  and  a  loss  of 
continuity  for  the  lost  distributions 

It  would  seem  important  to  always  have  in 
view  that  data  should  be  collected  at  the  source 
that  is  heavily  involved  in  the  generation  of 
these  data.   Medical  care  costs  associated  with 
rare  events  should  be  gathered  from  the 
institutions  which  provide  such  care,  and 
population  based  survey  frames  are  a  poor  source 
for  such  data.   It  is  obviously  important  to  get 
the  joint  distribution  of  income  levels  and 
income  sources,  and  of  sources  of  medical  care. 
Perhaps  it  would  be  more  efficient  to  add 
questions  about  the  type  and  source  of  medical 
care  to  an  already  existing  survey  with  a  long 
history  such  as  the  Current  Population  Survey  or 
the  Survey  of  Income  and  Program  Participation. 
During  a  time  of  rapid  transitions  in  the 
organization  of  medical  care  this  could  provide 
better  tracking  of  the  transition  than  could  be 
obtained  from  adding  detailed  income  questions 
to  the  HIS  Core.   In  general,  there  is  utility 
in  coordination  with  survey  activities  of 
Departments  other  than  Health  and  Human 
Services . 

Surveys  are  designed  to  have  samples  that 
represent  populations.   It  may  become 
increasingly  important  to  have  samples  designed 
with  subsets  valid  for  States  since  a  number  of 
functions  and  responsibilities  for  preventive 
care  and  medical  care  organization  are  or  will 
be  centered  in  the  States.   If  differences  in 
approach  to  the  organization  of  preventive  and 
medical  care  arise  between  States  it  would  be 
possible  and  important  to  monitor  differences  in 
outcome . 

If  the  Health  Interview  Survey  and  its 
sample  can  be  made  a  launching  platform  for  more 
specialized  studies,  and  longitudinal  panels, 
there  will  be  obvious  advantages  in  that  there 
will  be  linking  at  the  respondent  level.   A 
possible  problem  will  be  that  the  burden  on 
individual  respondents  will  inevitably  increase, 
with  some  likelihood  of  increased  dropout  rates, 
but  the  advantage  of  obtaining  joint 
distributions  of  additional  variables  is  an 
important  benefit. 

Many  European  countries,  and  especially 
the  Nordic  countries  have  in  place  a  central 
registration  system  which  constitutes  a  very 
rich  database  at  the  personal  level  addressable 
through  a  unique  lifetime  personal  identifier. 
A  large  number  of  studies  have  been  made 
possible  by  linking  these  data  sources  collected 
for  a  variety  of  purposes,  including  vital 
statistics,  civil  registries  of  residence, 
collection  of  taxes,  employment  records,  disease 
registries,  medical  care  records,  etc.   Such 
registries  contain  the  whole  universe  of  the 
population  and  obviously  can  be  used  to  answer 
questions  for  which  we  have  to  mount  special  or 
continuing  surveys.   It  is  clear  that  such  a 
universal  registration  system  cannot  be 
implemented  in  the  United  States  for  a  large 
number  of  reasons.   However,  it  would  seem  that 
::e   could  make  more  use  of  linking  individual 
records  collected  for  different  reasons  by 
different  institutions.   Especially  in  the  area 
of  health  care  organization  and  health  care 
costs  as  measured  in  claims  or  payments  very 
large  databases  exist  which  are  the  property  of 
State  and  Federal  Agencies,  and  similar 
databases  exist  in  the  private  sector  in  Health 
Maintenance  Organizations,  insurance  companies 


and  in  associations  of  hospitals  at  the  State 
level . 

It  might  be  useful  to  explore  how  we  could 
facilitate  and  promote  the  construction  of 
linked  records  that  are  stripped  of  all 
identifiers  after  having  been  linked  on  the 
basis  of  an  identifier  such  as  the  Social 
Security  Number  or  a  State  Identification  Number 
and  one  or  more  variables.   Such  studies  can  now 
be  done,  and  are  being  done,  but  usually  there 
are  substantial  barriers  to  overcome.   Many 
policy  questions  that  come  up  are  difficult  to 
address  with  statistical  surveys,  lend 
themselves  much  more  to  studies  based  on  already 
existing  data  that  have  to  be  linked  at  the 
record  level. 

In  1987  the  Public  Health  Committee  of  the 
Connecticut  Legislature  formally  asked  the 
Connecticut  Academy  of  Science  and  Engineering 
to  evaluate  the  Economic  Impact  of  AIDS  Health 
Care  in  Connecticut.   The  Connecticut  State 
Health  Department  maintains  a  registry  of 
persons  diagnosed  with  AIDS  as  a  part  of  the 
National  Registry,  and  the  then  Department  of 
Income  Maintenance  was  the  repository  for 
Medicaid  claims  and  payments.   The  two  files 
were  linked  within  the  State  Computer  Center  on 
a  State  Identifier  and  a  subset  of  all  Medicaid 
claims  and  payments  on  behalf  of  persons  with 
AIDS  was  produced  for  analysis.   The  State 
Health  Department  produced  a  file  of  persons 
diagnosed  with  AIDS  that  was  stripped  of  all 
identifying  information  other  than  the  State 
Identifier.   These  two  files  allowed  for  a 
comprehensive  analysis  of  Medicaid  supported 
cost  of  medical  care  of  persons  with  AIDS  in 
Connecticut.   It  was  found  that  49%  of  adult 
Persons  with  AIDS  had  their  medical  care 
reimbursed  by  Medicaid,  as  did  85%  of  pediatric 
patients.   This  percentage  was  slowly  increasing 
during  the  1980' s.   The  average  Medicaid  payment 
per  year  for  those  served  by  Medicaid  was 
$34,420  with  75%  of  that  in  the  form  of  hospital 
inpatient  charges,  19%  for  hospital  outpatient 
and  medical  payments  and  5%  for  drugs.   This  was 
an  example  of  providing  a  fairly  prompt  (1  year) 
answer  to  policy-relevant  questions  at  low  cost 
from  already  collected  data  through  record 
linking.  Without  the  ability  to  produce  a  linked 
file  it  would  have  been  very  difficult  to 
provide  the  estimate  requested.   It  was  not 
possible  to  deduce  the  AIDS  status  from  the 
Medicaid  records,  and  without  linking  with  the 
AIDS  register  no  reliable  estimates  could  be 
made . 

In  another  example  of  cooperative  research 
that  involved  access  to  a  State  Registry  for 
Birth  Defects  in  New  York  State  we  wished  to 
determine  whether  proximity  of  the  mother's 
residence  to  a  hazardous  waste  site  was  a  risk 
factor  for  birth  defects.   The  mother's 
residential  adddress  in  the  birth  certificate 
was  linked  to  the  TIGER  files  developed  by  and 
for  the  U.S.  Census  in  order  to  develop 
latitudes  and  longitudes  for  that  residence 
which  could  be  related  to  the  latitudes  and 
longitudes  that  could  be  derived  from  the 
Registry  of  Hazardous  Waste  Sites  for  New  York 
State  maintained  by  the  Department  of 
Environmental  Conservation.   The  resulting  file 
did  contain  all  the  information  from  the  Birth 
Defects  Registry  including  the  geographic 
coordinates,  and  all  the  information  from  the 
Registry  of  Hazardous  Waste  Sites  including  the 
characteristics  of  the  site  and  its  geographic 
coordinates.   As  a  result  we  could  determine  the 
association  of  different  birth  defects  with 
distance  from  a  hazardous  waste  site,  and  with 
the  characteristics  of  the  site.   The  use  of 
pre-existing  information,  combined  with  a 
facilitation  of  the  record  linking  in  a  manner 
that  protected  the  privacy  of  all  concerned  made 
it  possible  to  conduct  a  study  of  great 
statistical  power  which  detected  a  small  but 
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significant  associat 

I t  the  above  indicates  a  prejudice  about 
the  value  of  being  able  to  link  different  data 
sources,  I  confess.   In  the  discussions  that  I 
am  aware  of  that  are  concerned  with  improving 
the  efficiency  of  data  gathering,  and  the  need 
to  gather  more  data  for  research  in  support  of 
policy  evaluation  and  development,  there  is 
almost  no  discussion  of  the  possibility  of 
merging  existing  databases.   If  we  would  like  to 
tap  the  potential  for  this  type  of  linkage  which 
I  believe  to  be  substantial  it  will  be  necessary 
to  prepare  for  such  linkages  by  pursuing  the 
following,  not  necessarily  in  the  order 
presented  here: 


of  a  negotiation  and  implement 
would  include  input  from  const 
other  Departments  and  Agencies 
and  State  levels.  I  am  sure  t 
forward  to  an  improved  data  ga 
that  maintains  the  strengths  a 
and  at  the  same  time  anticipat 
needs  at  the  Federal  and  State 
recognizes  the  benefits  of  mor 
bases  at  the  record  level. 


at  ion  phase  that 
ituencies  and 

at  the  Federal 
hat  all  of  us  look 
thering  effort 
lready  established 
es  policy  research 

level  and 
e  linking  of  data 


■  in  surveys  or  data  sets  that  are  to  be  linked 
there  needs  to  be  a  key  on  which  such  links 
can  be  made.   In  the  U.S.  the  Social  Security 
Number  suggests  itself,  although  there  may  be 
some  problems  with  that 

■  HHS  may  find  it  useful  to  produce  and  make 
available  in  one  document  codebooks  of  all 
surveys  or  data  sets  that  contain  data  that 
might  be  relevant  to  health  care  policy  or 
public  health  preventive  potentials 

■  there  needs  to  be  agreement  on  a  protocol 
that  provides  effective  assurance  that  no 
individual  whose  data  are  already  recorded  in 
one  of  the  files  being  linked  will  be  able  to 
be  identified  in  the  linked  file  or  at  any 
time  in  the  process 

■  there  may  need  to  be  an  agreement  about  the 
computer  facility  that  will  carry  out  a 
linkage  and  produce  the  linked  file, 
especially  if  files  from  different  agencies 
or  institutions  need  to  be  linked 

The  announced  intent  to  integrate, 
consolidate  or  merge  different  HHS  surveys  by 
using  the  HIS  as  a  sampling  frame  to  mount  a 
number  of  more  specialized  surveys,  while  taking 
advantage  of  the  NHIS  Core  data  is  in  principle 
a  sound  way  of  using  linking  of  surveys,  in  this 
case  linking  through  selection  of  participants 
for  other  surveys  from  the  HIS  Core,  based  on 
characteristics  recorded  in  that  core. 

Without  intending  to  be  critical  of  what 
was  clearly  an  effort  under  extreme  time 
constraints  and  strong  directives  it  is  worth 
mentioning  the  claimed  benefits  and  liabilities 
that  were  perceived  by  the  participants  in  the 
process.   The  benefits  claimed  were: 

■  coordination  of  efforts 

■  filling  in  of  gaps  in  coverage 

■  potential  of  linked  surveys 

■  reduced  respondent  burden 

■  improved  efficiency  in  all  parts  of  the 
survey  process 

■  improved  ability  to  focus  on  high-priority, 
data  needs 

Among  the  liabilities  that  were  considered 
in  the  Survey  Integration  Plan: 

■  Time  pressure  in  implementation 

■  Larger  overall  budget:  vulnerable  for 
reductions 

■  Interrelatedness  increases  vulnerability  to 
breakdowns  of  any  one  element 

■  It  may  be  more  difficult  to  accomodate 
demands  for  special  analyses 

It  would  appear  that  the  benefits  are 
items  that  are  objectives  to  be  pursued  without 
any  guarantee  of  being  achieved,  whereas  the 
liabilities  are  likely  to  stare  us  in  the  face 
from  the  start.   This  observer  would  urge  that 
the  product  of  the  task  force  working  on  the 
integration  of  HHS  Surveys  be  the  starting  point 
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1  Introduction 

We  describe  a  set  of  exploratory  tools  to  investigate  and 
to  model  spatial-temporal  patterns  of  diseases.  The  essential 
idea  is  to  integrate  statistical  and  visual  modelling  with  in- 
teractive visual  representation.  The  disease  data  represents 
multiple  spatial-temporal  processes  where  each  reported  ob- 
servation (number  of  cases  in  space-time  region)  is  an  aggre- 
gate quantity  (averaged  over  the  particular  region).  The  im- 
plemented visual  representations  of  such  data  include  static 
and  dynamic  maps  and  time  series  plots.  The  modelling 
tools  include  transformations,  color  mappings,  smoothing, 
aggregation,  estimation  of  dependence  structure,  and  other 
models.  The  toolset  can  be  easily  extended  by  a  non-expert 
using  the  S  language  (Becker,  Chambers,  and  Wilks,  1988) . 
We  stan  by  describing  the  data,  then  describe  our  approach 
and  some  features  of  the  implementation. 

2  Data 

The  data  we  use  were  obtained  from  the  Centers  for 
Disease  Control  and  Prevention  (CDC)  which  operates  the 
National  Notifiable  Diseases  Surveillance  System  (NNDSS) 
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Table  1:  The  list  of  diseases  and  numbers  of  cases 

in  partnership  with  the  Council  of  State  and  Territorial  Epi- 
demiologists (CSTE).  The  CDC  collects  weekly  provisional 
information  on  the  occurrence  of  diseases  that  are  defined  as 
"notifiable"  by  CSTE.  Further  details  concerning  the  NNDSS 
can  be  found,  for  example,  in  Chorba  et  al.  (1989). 

The  dataset  contains  weekly  by  state  reports  on  57  dis- 
eases for  the  period  between  1980  and  1994.  There  are  783 
report  weeks  in  this  period  and  the  reports  are  provided  for 
5 1  states,  3  territories  and  New  York  City.  The  names  of  the 
reported  diseases,  the  total  number  of  cases  over  the  reported 
period,  and  the  number  of  missing  reports  are  in  Table  1 . 
The  table  conveys  a  general  idea  of  how  widespread  each 


particular  disease  is  and  how  much  care  is  taken  to  report  the 
disease  cases. 

3     Multivariate  Interactive  Animation 
System  For  Map  Analysis 

We  previously  analyzed  similar  data  on  the  disease  mumps 
in  Eddy  and  Mockus  (1993).  The  analysis  we  performed  pro- 
duced successively  smoother  non-interactive  dynamic  maps 
of  the  incidence  rates  and  a  dynamic  map  of  the  residuals 
from  a  two  way  analysis  of  variance  model.  Here  we  gener- 
alize this  approach  to  multiple  diseases.  We  have  designed 
a  system  to  integrate  dynamic  and  static  maps,  transforma- 
tions, smoothing,  and  other  techniques  for  spatio-temporal 
modelling  and  visualization.  We  refer  to  this  system  as  MI- 
ASMA (for  Multivariate  Interactive  Animation  System  For 
Map  Analysis). 

The  system  accepts  observations  in  the  form  of  a  three 
dimensional  array.  The  first  dimension  ranges  over  all  re- 
gions in  space  (e.g.,  counties  or  states).  The  second  dimen- 
sion ranges  over  all  time  periods  (e.g.,  days,  weeks,  months, 
years).  The  third  dimension  lists  all  the  quantities  of  inter- 
est (e.g.,  population  size,  number  of  cases  for  a  particular 
disease,  incidence  rates  for  a  particular  disease,  or  derived 
quantities).  Notice,  that  we  only  consider  quantities  that  are 
averages  over  regions  in  space  and  time.  The  first  two  di- 
mensions of  the  array  define  a  dynamic  map  and  the  last  one 
implies  that  there  may  be  multiple  dynamic  maps. 

The  input  to  the  system  can  be  either  original  reported 
data  or  the  output  from  a  statistical  model  (e.g.  residuals 
and/or  effects  of  a  spatial  temporal  model).  Those,  we  can 
successively  fit  models  and  inspect  residuals  until  we  obtain 
a  satisfactory  fit. 

The  system  is  implemented  as  a  selection  of  different 
classes  of  tools.  The  most  important  classes  include  multiple 
visual  representation,  transformation,  handling  of  missing 
values,  aggregation  and  smoothing,  superpositions  of  several 
quantities,  and  statistical  model  fitting  toolsets. 

We  separate  modelling  tools  into  two  groups:  models 
for  visual  representation,  and  models  for  statistical  analy- 
sis. The  models  for  visual  representation  must  be  tightly 
integrated  into  the  system  to  allow  adequate  interactive  re- 
sponse, while  statistical  models  can  be  loosely  integrated  via 
file  sharing.  We  separate  the  implementation  of  modelling 
and  visualization  tools  to  simplify  extensions  to  the  system. 

3.1     Display 

An  example  display  of  our  system  is  given  in  Figure  1 . 

The  system  consists  of  the  main  control  window  and  vari- 
ous view  windows.  The  control  window  contains  menus  and 
selection  lists.  Modelling  and  transformation  methods  are 
controlled  from  the  main  window.  In  Figure  1  the  data  on 
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the  disease  aseptic  meningitis  is  selected  and  the  rank  trans- 
formation is  being  used.  The  main  window  also  contains 
the  current  date  (year  and  week)  for  the  dynamic  map  view 
shown  at  the  bottom.  A  time  series  plot  of  the  disease  inci- 
dence in  Alabama  is  in  the  window  overlapping  the  control 
window.  The  state  and  the  disease  can  be  selected  interac- 
tively using  scrollbars  at  the  bottom  and  at  the  left  of  the  time 
series  plot. 

3.2    Visual  Models 

The  models/tools  for  visual  representation  include  view 
selection,  spatial  smoothing,  time  interpolation,  transforma- 
tion, color  mapping,  display  of  missing  values,  and  display 
of  multiple  quantities.  The  available  views  are  time  series, 
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Figure  1:  Control  window  and  two  views  of  the  MIASMA 


static  maps,  and  dynamic  maps.  Each  type  of  view  is  pre- 
sented in  a  separate  window  and  an  unlimited  number  of 
windows  can  be  created  for  each  view. 

The  time  series  view  presents  the  time  series  of  the  data 
for  a  particular  quantity  and  spatial  region  over  the  available 
time  period.  Two  scrollbars  allow  selection  of  the  spatial 
region  and  quantity  of  interest. 


provided.  The  simplest — piecewise  constant  interpolation 
shows  a  constant  value  within  each  region  and  within  each 
time  interval.  A  slightly  smoother  animation  can  be  ob- 
tained by  interpolating  linearly  between  two  time  intervals, 
namely,  two  maps  are  computed  for  each  time  period  as  the 
key  frames  and  then  linearly  interpolated  for  the  interme- 
diate frames.  The  number  of  intermediate  frames  can  be 
changed  to  increase  or  decrease  the  speed  of  the  animation. 
A  spatially  smooth  animation  can  be  obtained  by  spatial  in- 
terpolation as  described  in  Eddy  and  Mockus  (1993).  The 
smoothing  parameter  and  starting  location  for  the  animation 
can  be  selected  using  scrollbars. 

We  consider  spatial  and  temporal  smoothing  as  a  visual 
model  since  it  facilitates  perception  of  the  dynamic  map. 
Time  smoothing  removes  jumps  in  time  (the  jumps  create 
a  distracting  blinking  appearance)  and  smoothing  in  space 
hides  region  boundary  artifacts  (which  are  clearly  perceived 
by  an  observer  but  are  often  irrelevant  to  the  quantity  of  inter- 
est). Time-space  smoothing  represents  a  simple  visual  model 
that  allows  qualitative  (without  excessive  detail)  display  of 
data  that  was  aggregated  over  regions  in  time  and  space.  We 
have  found  that  in  a  dynamic  display  it  is  important  to  limit 
the  amount  of  information  since  it  is  not  trivial  to  perceive 
every  detail  even  with  the  high  bandwidth  of  a  human  visual 
system.  Smoothing  can  be  viewed  as  fitting  of  a  statistical 
model;  we  discuss  that  approach  later. 

Essential  visual  representation  models  are  various  trans- 
formations of  the  data  into  the  range  of  available  display 
attributes.  The  simplest — linear  transformation — can  often 
be  inadequate  due  to  the  discrete  nature  and  limited  range  of 
display  attributes,  such  as,  pixels,  colors,  patterns.  We  have 
found  that  different  transformations  emphasize  different  fea- 
tures present  in  the  data,  e.g.,  few  large  outliers  can  make  a 
dynamic  map  look  almost  constant  if  a  linear  transformation 
is  used.  Rank  transformation,  on  the  other  hand,  ignores  the 
outliers  and  make  the  distribution  of  the  display  attributes 
uniform  over  the  available  range.  In  addition  to  linear  and 
rank  transformations  we  have  arbitrary  power  transforma- 
tions (extended  by  a  logarithmic  transformation).  In  the  case 
of  dynamic  and  static  maps  we  encode  the  value  of  a  quantity 
of  interest  by  color.  The  transformation  (mentioned  above) 
converts  each  value  into  an  integer  code  in  the  range  between 
1  and  256  (code  0  is  reserved  for  the  background).  Those 
codes  are  then  displayed  according  to  the  colormap  that  maps 
each  code  into  a  color.  The  colormap  can  be  selected  interac- 
tively. For  a  discussion  on  how  to  select  colormaps  to  convey 
quantitative  information  see  Levkowitz  and  Herman  (1992). 


The  static  map  view  displays  a  static  map  of  the  data 
where  color  of  a  pixel  in  the  window  encodes  the  value  of  the 
quantity  at  the  particular  location  in  time  and  space.  Since 
the  quantities  are  aggregates  over  a  space-time  region  we 
provide  several  interpolation  techniques.  The  simplest  one 
just  shows  a  constant  value  within  each  region.  As  with  a 
time  series  view  the  two  scrollbars  allow  selection  of  the 
quantity  to  be  displayed  and  the  time  moment. 

The  dynamic  map  view  is  an  animation  of  a  static  map. 
As  with  the  static  map,  several  interpolation  techniques  are 


3.2.1     Display  of  Missing  Values 

By  inspecting  Table  1  it  becomes  apparent  that  a  sub- 
stantial amount  of  observations  are  missing.  It  is  essential  to 
address,  this  problem  in  constructing  a  visual  representation 
of  the  missing  data. 

We  implement  a  number  of  ways  to  address  this  problem. 
In  the  time  series  view  we  show  the  data  as  small  dots. 
Missing  data  is  absent  from  the  display,  although  we  can 
infer  its  presence  from  the  larger  horizontal  gaps.  We  take 
two  different  approaches  in  the  case  of  static  and  dynamic 
maps.    The  two  alternatives  are  to  leave  the  missing  data 
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out  (use  a  neutral,  background,  transparent,  or  some  other 
color  that  is  not  present  in  the  color  scale)  or  to  fill  in  some 
color  using  the  available  data.  Currently  we  show  missing 
values  in  the  background  color  or  impute  the  color  (by  taking 
a  median  value  for  each  time  moment)  and  add  a  pattern  to 
indicate  that  that  value  was  not  observed. 

3.3    Statistical  models 

We  have  found  several  statistical  models  very  useful  to 
study  these  data.  We  start  from  the  simplest  but  very  useful 
models  of  aggregation,  then  consider  smoothing  methods,  a 
two-way  table,  estimation  of  dependence  structure,  and  best 
linear  prediction  (kriging). 

3.3.1     Aggregation 

Various  aggregation  methods  can  dramatically  reduce 
the  amount  of  data  and  simplify  the  inspection  process.  By 
aggregation  we  mean  reduction  of  the  number  of  observa- 
tions in  our  dataset  (represented  by  a  3-dimensional  array 
(diseases,  time  moments,  states))  by  combining  several  cells 
into  a  single  cell.  The  aggregations  differ  in  which  cells 
are  selected  for  aggregation  (neighborhoods)  and  in  which 
method  is  used  to  produce  a  single  value  out  of  values  in 
the  neighborhood.  Another  operation  which  we  call  smooth- 
ing operates  the  same  way  as  aggregation  except  it  does  not 
reduce  the  number  of  cells  in  the  data  array,  i.e.,  for  each 
cell  a  neighborhood  of  cells  is  defined  and  then  the  combine 
operator  is  applied  to  the  values  in  the  neighborhood  of  a  cell 
to  produce  a  single  value  for  that  cell.  Since  the  smoothing 
and  aggregation  methods  are  so  similar  we  will  consider  only 
aggregation  methods.  Analogous  selection  of  neighborhoods 
and  combine  operators  is  available  for  smoothing  operations 
too. 

Definition  of  neighborhoods.  We  consider  several  ways  to 
select  neighborhoods  by  selecting  a  direction  in  the  data  array 
and  by  selecting  the  size  of  the  neighborhood.  Since  the  data 
is  represented  by  a  3-dimensional  array  xitj<k  the  aggregation 
direction  is  defined  by  selecting  the  index  of  the  array.  The 
first  index  corresponds  to  spatial  location,  the  second  -  to 
time  intervals,  and  the  third  -  to  diseases. 

For  example,  we  can  aggregate  over  time  intervals  with 
the  window  size  of  4  time  intervals  to  convert  weekly  data  to 
monthly  data.  The  monthly  data  array 

Xi,J,k  =  Agg(a;ll4J)A:,X,?4J+j>«;)a;,i4j+2lfc,  £;,4./+3,fc)i 

where  Xijik  is  weekly  data  array,  and  Agg()  is  aggregation 
operator.  The  aggregation/smoothing  operator  can  be  se- 
lected independently  of  the  neighborhood  selection.  It  can 
range  from  a  simple  sum  (Agg(i,  y,  z,  w)  =  x  +  y  +  z  +  w) 
to  an  ARIMA  filter,  where  the  result  represents  parameter 
values  of  the  ARIMA  process,  or  the  smoothed  version  of  of 
the  time  series. 

We  can  investigate  periodic  behavior  in  the  dataset  by 
defining  appropriate  neighborhoods.  For  example,  to  pro- 
duce a  standard  yearly  cycle  from  weekly  data  we  can  aggre- 
gate values  for  a  particular  week  over  all  years. 


Spatial  neighborhoods  need  be  treated  differently  from 
time  neighborhoods.  To  define  spatial  neighborhoods  we 
need  to  define  adjacencies  between  the  locations  of  obser- 
vations because  the  simple  ordering  by  time  is  no  longer 
present.  In  our  case  regions  A;  (states)  form  a  partition  of 
A  (the  continental  US).  We  define  two  spatial  regions  to  be 
adjacent  (or  one-adjacent)  if  they  share  a  common  border 
consisting  of  more  than  one  point.  If  there  is  a  region  to 
which  they  both  are  adjacent  then  we  call  them  two-adjacent. 
Similarly  we  can  define  k-adjacent  regions.  The  sizes  of  the 
spatial  neighborhood  is  the  number  k. 

It  may  be  of  interest  to  aggregate  over  different  quantities 
(diseases)  in  an  attempt  to  capture  relationships  between  dif- 
ferent diseases.  Any  composition  of  aggregation/smoothing 
methods  can  be  performed  within  MIASMA. 

Aggregation  operators  can  be  divided  into  several  classes: 
arithmetic,  order,  selection,  composition,  and  other.  The 
arithmetic  operators  include  sum  and  variance,  the  order  op- 
erators include  various  quantiles,  the  selection  operators  se- 
lect a  value(s)  based  on  position  within  the  neighborhood 
(section,  several  sections).  Composition  of  the  operators  is 
also  possible.  More  complicated  complicated  operators  are 
described  in  the  next  section. 

3.3.2     Other  statistical  models 

Given  a  complicated  structure  of  observations  it  seems 
useful  to  be  able  to  inspect  the  model  and  the  residuals.  For 
example,  let  zijk  be  the  reported  incidence  rates  of  disease  k  in 
state  i  for  month  j.  We  can  use  median  polishing  to  fit  a  model 
Zijk  =  Sik  +  tjk  +  Vijk,  where  s.^'s  are  state-disease  effects 
and  tjk's  are  time-disease  effects.  Since  the  model  contains 
a  large  number  of  parameters  we  may  inspect  the  parameters 
as  well  as  the  fit  using  the  tools  available  in  MIASMA.  To 
do  that  we  can  use  derived  datasets  Effects,^  =  s,-*  +  tjk  or 
Residuals,^  =  77,^  instead  of  the  original  observed  incidence 
rates.  In  particular,  the  residuals  do  not  have  seasonal  and 
longer  term  trends  observed  in  the  disease  reports,  but  can 
indicate  unusually  high  incidence  rates  (epidemics). 

Another  approach  is  to  model  the  disease  as  a  space-time 
process.  The  observed  data  would  represent  the  integrals 
of  the  incidence  rate  process  over  the  regions  Aij  in  space 
and  time,  Xijk  =  fA.  fk(x)dx.  To  predict  the  process  fk  at 
a  particular  location  in  space  and  time  one  has  to  estimate 
(or  know)  the  covariance  function  of  the  process  fk.  For  a 
reference  on  spatial  prediction  see  Cressie  (1991).  Estimation 
of  the  spatial  covariance  function  from  aggregate  data  is 
described  in  Mockus  (1994). 

4     Summary 

We  designed  and  implemented  a  system  (MIASMA) 
to  analyze  spatio-temporal  patterns  in  a  dataset  containing 
weekly  reports  of  57  diseases  in  the  United  States.  The 
system  integrates  statistical  and  visual  representation  tools 
for  interactive  modelling  and  exploratory  analysis  of  simi- 
lar datasets.  The  visual  representation  tools  can  be  used  to 
look  at  the  raw  data,  at  the  fitted  models,  or  at  the  residu- 
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als  from  the  fitted  models.  We  found  that  interactive  model 
fitting  and  exploratory  analysis  is  essential  in  dealing  with 
large  spatial  dataset.  Our  modeling  and  visualization  tools 
are  geared  to  analyze  aggregate  data,  when  observations  rep- 
resent averages  over  space-time  regions  of  some  underlying 
process.  In  particular  we  implement  smoothing,  interpo- 
lation, aggregation,  and  prediction  methods  for  aggregate 
space-time  dataset.  We  separated  implementation  of  the  vi- 
sual representation  and  statistical  analysis  tools  to  simplify 
extensions  of  our  system. 
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"Violence  is  a  public  health  problem." 
That  statement  and  the  description  of  the  scope 
of  the  problem  have  become  grimly  familiar  to 
most  of  us.  In  1992,  there  were  over  37,000 
firearm-related  deaths  in  the  U.S. --and  firearms 
have  become  the  leading  or  second  leading  cause 
of  injury  death  in  at  least  15  states.  However, 
firearm-related  deaths  tell  only  part  of  the 
story;  for  every  firearm-related  death,  there 
are  many  other  violence-related  injuries.  Based 
on  emergency  department  surveillance,  we 
estimate  that  there  are  over  100  other  violence- 
related  injuries  of  any  weapon  type  for  every 
firearm-related  death  in  Massachusetts. 

A  public  health  approach  to  a  disease  or 
condition  resulting  in  excess  or  premature 
mortality  and  morbidity  encompasses  at  least 
three  components:  surveillance,  intervention, 
and  evaluation.  Our  ability  to  develop  and 
evaluate  violence  prevention  interventions  has 
been  limited  by  the  lack,  of  adequate 
surveillance  data.  Due  to  the  limitations  of  our 
data  systems,  we  typically  have  been  unable  to 
characterize  the  nature  of  nonfatal  injuries. 
The  Massachusetts  Weapon-Related  Injury 
Surveillance  System  (WRISS)  was  developed  to 
address  these  limitations (1 ,  2)  . 

The  surveillance  system  first  was 
implemented  as  a  pilot  program  in  1989,  with 
funding  from  the  Centers  for  Disease  Control. 
That  pilot  program  has  been  expanded  into  a 
statewide  surveillance  system,  with  reporting 
from  all  85  hospital  emergency  departments  in 
the  Commonwealth.  Today's  presentation  will 
summarize  the  first  year  of  statewide 
surveillance  data  and  compare  these  data  to 
mortality  data. 

System  Description  and  Methods 

Since  1927,  all  Massachusetts  physicians 
have  been  required  to  report  all  gunshot  wounds 
(GSWs)  and  violence-related  sharp  instrument 
wounds  (SIWs)  to  state  and  local  law  enforcement 
authorities.  Over  40  other  states  have  similar 
reporting  requirements.  Although  the 
Massachusetts  statute  requires  reporting  by 
physicians,  WRISS  is  a  voluntary,  emergency 
department -based  reporting  system  in  which 
hospitals  also  report  these  injuries  to  the 
Massachusetts  Department  of  Public  Health 
(MDPH) .  WRISS  reporting  incorporates  existing 
statutory  requirements  with  additional  data 
elements.  Based  on  periodic  emergency  department 
record  audits,  system  reporting  compliance  is 
estimated  to  be  between  70-80%,  and  there  is  no 
apparent  systematic  reporting  bias.  This 
compliance  estimate  is  significantly  higher  than 
the  30%  estimated  compliance  to  law  enforcement 
authorities  prior  to  WRISS  implementation. 

The  system  has  enabled  us  to  describe  the 
number  and  nature  of  nonfatal  weapon-related 
injuries,  to  compare  these  injuries  to  fatal 
injuries,  and  to  describe  differences  among 
population  groups  and  geographic  areas.  In 
addition,  the  system's  findings  are  critically 
important  to  the  development  of  local  violence 


prevention  initiatives.  The  methodology  of  the 
system  was  developed  in  a  way  that  would 
facilitate  its  replication  or  modification  by 
other  states  or  localities. 

Findings 

During  calendar  year  1994,  there  were  2865 
weapon-related  injuries  to  Massachusetts 
residents  reported  to  the  MDPH:  980  were  GSWs 
and  1885  were  violence-related  SIWs.  By 
definition,  all  GSWs  and  all  violence-related 
SIWs  treated  in  Massachusetts  hospital  emergency 
departments  are  reportable.  Excluded  are  any 
injuries  treated  outside  of  an  ED,  such  as  in  a 
physician's  office  or  clinic,  and  any  fatal 
cases  not  taken  to  an  ED.  Unless  otherwise 
noted,  numbers  and  rates  presented  here  are 
based  on  reported  cases  not  adjusted  for 
compliance.  Therefore,  the  actual  number  of 
cases  and  rates  are  somewhat  higher. 

Violence-Related  Gunshot  Wounds:  Of  the 
980  GSWs  reported  to  WRISS,  662  or  68%  were 
violence-related,  17%  were  unintentional  or 
"accidental,"  5%  were  intentionally  self- 
inflicted,  and  11%  were  of  unknown  intent.  The 
rate  of  violence-related  GSWs  was  11  per  100,000 
population  compared  to  2.7  for  unintentional 
GSWs. 

Most  victims  were  male  (90%,  n=598), 
although  males  comprise  only  48%  of  the 
population.  Half  of  the  victims  (n=328)  were 
black,  nonHispanic  compared  to  5%  of  the 
population.  Of  the  remaining  cases,  18%  (n=121) 
were  white,  nonHispanic  and  24%  (n=157)  were 
Hispanic.  Race  was  missing  or  unknown  in  about 
6%  (n=41)  of  cases.  GSW  victims  were  young, 
often  very  young.  Sixty-three  percent  (n=414) 
were  under  the  age  of  25,  with  teens  and  young 
adults  (15-24)  accounting  for  60%  (n=394)  of 
cases.  The  injury  rate  for  this  group  was  seven 
times  higher  than  the  rate  for  adults  over  24 
years  old  (42.2  vs  5.7  per  100,000).  Within  the 
15-24  age  group,  teens  had  a  higher  injury  rate 
than  the  20-24  year  olds  (47.4  vs  38.1,  n=197 
for  each) . 

The  disposition  of  ED  patients  highlights 
the  need  for  an  ED  based  surveillance  system.  A 
quarter  (27%,  n=l79)  were  treated  and  discharged 
from  the  ED;  57%  (n=379)  were  admitted  to  the 
hospital;  9%  died  in  the  ED  (n=59) ;  the  status 
of  7%  (n=45)  of  victims  was  missing  or  unknown. 
Without  an  ED  reporting  system,  we  would  have 
little,  if  any,  information  on  the  25-30%  of  GSW 
victims  treated  and  discharged.  In  addition, 
without  E-coding  of  hospital  discharge  data, 
little  would  be  known  about  the  57%  of  patients 
admitted  to  the  hospital .  Although  E  codes  have 
been  mandated  in  Massachusetts  beginning  in 
1994,  these  data  still  are  unavailable  for 
analysis.  In  addition,  the  quality  of  E-coded 
data  is  unknown,  and  there  is  a  significant 
time-lag  between  injury  occurrence  and  data 
availability. 

The  development  of  violence  prevention 
initiatives  requires  knowledge  about  the 
circumstance  of  the  injury,  the  relationship 
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between  the  victim  and  offender,  and  the  place 
of  injury  occurrence.  For  example,  very 
different  prevention  strategies  might  be 
employed  for  drive-by  shootings  in  contrast  to 
domestic  violence.  This  information 
traditionally  has  been  difficult  to  obtain,  and 
it  also  has  been  difficult  to  obtain  as  part  of 
ED  surveillance.  Almost  two-thirds  (63%,  n=354) 
of  violence-related  GSW  cases  did  not  include 
victim-offender  relationship;  data  were  missing, 
unknown,  or  the  patient  was  unwilling  to 
disclose  the  relationship.  In  26%  of  cases,  the 
offender  was  reported  as  a  stranger,  and  in  11% 
of  cases  the  offender  was  known  to  the  victim. 
Reporting  for  the  circumstance  of  the  injury  was 
only  slightly  better  with  54%  missing  or  unknown 
(n=3  02) .  For  cases  in  which  circumstance  was 
reported,  29%  (n=75)  were  related  to 
argument/abuse,  39%  (n=102)  were  crime-related, 
and  32%  (n=84)  were  reported  as  other 
circumstances.  Fifty-four  percent  (n=359)  of 
GSWs  were  reported  as  occurring  on  the  street, 
13%  in  a  house,  14%  in  other  locations.  Location 
was  missing  or  unknown  in  19%  (n=124)  of  cases. 

Given  the  large  proportion  of 
unknown/missing  data,  these  findings  must  be 
interpreted  with  caution.  However,  the  results 
are  suggestive  of  areas  for  further  study.  It 
may  be  that  more  qualitative,  in-depth  research 
is  necessary  to  understand  these  variables. 
Surveillance  data  may  be  most  useful  for 
suggesting  areas  in  which  additional  research 
should  be  conducted. 

Unintentional  Gunshot  Wounds:  The 
development  of  WRISS  was  motivated  largely  by 
the  need  to  learn  more  about  violence-related 
injuries.  However,  because  the  law  requires  the 
reporting  of  all  GSWs,  important  data  on 
unintentional  injuries  also  have  been  collected 
through  WRISS.  Because  most  self-inflicted  GSWs 
are  fatal  and  are  not  treated  in  the  ED,  the 
surveillance  system  adds  relatively  little  to 
our  understanding  of  these  injuries.  Among  the 
980  reported  GSWs,  163  or  17%  were  unintentional 
or  "accidental."  Children  and  youth  under  20 
years  old  were  represented  disproportionately 
among  unintentional  injury  cases  (50%) . 
Unintentional  GSWs  tended  to  be  less  severe  than 
violence-related  ones;  only  20%  (n=32)  of  these 
victims  were  admitted  to  the  hospital,  compared 
to  57%  of  violence-related  GSW  victims. 

A  partial  explanation  for  the  difference 
in  severity  was  somewhat  surprising,  namely 
weapon  type.  More  than  half  (56%)  of  the 
unintentional  gun  injuries  were  caused  by  BB  or 
pellet  guns.  Forty-seven  percent  of  these 
injuries  were  to  children  under  the  age  of  15. 
Because  these  injuries  rarely  result  in  hospital 
admission  or  death,  very  little  has  been  known 
about  them.  In  fact,  they  often  are  perceived  as 
minor,  and  BB  or  pellet  guns  frequently  are  used 
by  children  with  the  knowledge  and  consent  of 
adults.  However,  these  guns  can  cause  blindness 
and  other  eye  injuries.  Among  Massachusetts 
residents,  a  quarter  of  injuries  from  these  guns 
were  to  the  face  or  head.  Newer,  more  powerful, 
BB  and  pellet  guns  are  capable  of  even  more 
serious  injury- -and  nationally  there  are  several 
fatalities  each  year.  Thus,  WRISS  was  able  to 
identify  a  previously  hidden  source  of  gun- 
related  injury  for  which  prevention  initiatives 
can  be  developed.  Although  BB  gun  injuries  were 


more  common  in  small  communities,  it  is 
interesting  to  note  that  in  both  large  and  small 
communities  about  23%  of  BB  gun  injuries  were 
violence-related.  Despite  a  popular  perception 
that  these  are  toys  or  sporting  guns,  they  are 
being  used  as  weapons  in  interpersonal  violence. 

Violence-Related  Sharp  Instrument  Wounds: 
The  focus  of  this  presentation  has  been 
primarily  on  violence-related  GSWs.  However, 
WRISS  is  unique  in  that  it  also  collects 
information  on  violence-related  SIWs. 
Information  about  SIWs  is  even  less  available 
than  about  GSWs  because  most  analyses  of 
violence-related  injuries  rely  on  mortality 
data,  and  relatively  few  SIWs  result  in  death. 
SIWs  are  a  significant  part  of  intentional 
injury,  and  summary  data  are  presented  here  to 
supplement  the  findings  on  GSWs.  Almost  three 
times  as  many  violence-related  SIWs  (n=1885)  as 
GSWs  (n=662)  were  reported  to  the  MDPH.  Until 
the  development  of  WRISS,  only  the  crudest  of 
estimates  of  the  number  of  nonfatal  SIWs  had 
been  available. 

Most  SIW  victims  were  male  (83%,  n=1566), 
although  female  victims  were  slightly  more 
common  than  for  GSWs  (15%  vs  7%) .  About  a  third 
of  SIW  victims  were  black,  nonHispanic  (31%, 
n=586),  about  a  third  were  white,  nonHispanic 
(36%,  n=687) ,  and  19%  (n=355)  were  Hispanic. 
Thus,  there  was  considerately  more  racial/ethnic 
variation  among  victims  of  SIWs  than  among 
victims  of  GSWs,  although  black  and  Hispanic 
victims  still  were  overrepresented.  The  age 
distribution  of  SIW  victims  was  similar  to  that 
of  GSW  victims  in  that  there  was  a  peak  in  the 
teen/young  adult  age  groups.  However,  SIW 
victims  were  somewhat  older,-  42%  (n=790)  were 
under  the  age  of  25  compared  to  63%  of  GSW 
victims.  SIWs  tended  to  be  less  severe  than 
GSWs.  Less  than  a  third  (30%,  n=561)  were 
admitted  to  the  hospital,  55%  (n=1041)  were 
treated  in  the  ED  and  discharged,  and  14% 
(n=265)  of  cases  had  a  missing  or  unknown 
disposition. 

SIW  victims  were  injured  in  a  wider 
variety  of  places,  under  more  varied 
circumstances,  and  were  more  likely  to  report 
being  injured  by  someone  they  knew  than  GSW 
victims.  In  cases  where  victim-offender 
relationship  was  reported,  57%  (n=547)  involved 
persons  known  to  each  other,  and  in  43%  (n=408) 
the  offender  was  unknown  to  the  victim.  In  41% 
(n=656)  of  cases  victim-offender  relationship 
was  reported  as  unknown.  Victim-offender 
relationship  was  more  likely  to  be  reported  for 
SIWs  than  for  GSWs  (59%  vs  37%) .  The 
circumstances  of  SIWs  were  different  from  those 
reported  for  GSWs ;  44%  (n=710)  resulted  from  an 
argument  or  abuse,  compared  to  13%  for  GSWs;  15% 
(n=247)  were  crime-related;  and  a  smaller 
proportion  reported  missing/unknown  (33%,  n=485) 
than  for  GSWs  (54%) . 

Advantages  of  a  Statewide  System 

Statewide  surveillance  permits  analyses  of 
weapon-related  injury  data  in  ways  that 
previously  were  not  possible.  For  example,  we 
now  are  able  to  calculate  and  compare  rates  on 
both  a  statewide  and  local  level.  The 
availability  of  nonfatal  statewide  data  has 
enabled  us  to  compare  morbidity  and  mortality 
data  and  to  calculate  case  fatality  rates. 
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Demographic  differences  (by  age,  race, 
sex)  in  violence-related  GSW  rates  already  have 
been  mentioned.  Rates  also  varied  by  size  of 
community.  Almost  79%  (n=521)  of  GSW  victims 
lived  in  large  communities  (50K+)  compared  to 
37%  of  the  population.  Firearm-related  injuries 
were  more  of  a  large  city  problem,  primarily 
because  of  violence-related  injuries.  The  rate 
of  GSWs  in  large  communities  was  30  per  100,000 
compared  to  6  per  100,000  in  small  communities- - 
with  a  rate  ratio  of  large/small  of  5.  That  is, 
the  risk  for  a  GSW  was  five  times  higher  in  a 
large  than  in  a  small  community.  Rates  steadily 
increased  with  community  size,  with  Boston,  the 
State's  largest  city,  having  a  rate  of  47  per 
100,000.  Boston  cases  accounted  for  almost  half 
of  the  total  GSW  reports.  Coupled  with  the  data 
on  age,  race,  and  sex  of  victims,  we  can  see 
that  young,  minority  urban  males  were 
represented  disproportionately  among  victims  of 
violence-related  GSWs. 

Through  WRISS  we  also  have  determined  that 
while  violence-related  gun  injuries  tended  to 
cluster  in  large  cities,  unintentional  injuries 
were  distributed  more  generally  throughout  the 
state.  The  rate  of  accidental  injuries  was 
similar  in  large  and  small  communities. 

Morbidity  and  Mortality 

A  statewide,  population-based  surveillance 
system  for  weapon-related  injuries  permits  the 
comparison  of  fatal  and  nonfatal  injuries  using 
both  surveillance  and  vital  statistics  data. 
Most  analyses  of  weapon-related  injuries  have 
focused  on  mortality  due  to  the  availability  of 
vital  statistics  or  police  homicide  data  and  the 
lack  of  morbidity  data.  WRISS  has  revealed  some 
important  differences  in  the  epidemiology  of 
fatal  and  nonfatal  injuries. 

For  the  comparison  of  mortality  and 
morbidity  data,  nonfatal  cases  from  WRISS  and 
fatal  cases  from  the  MDPH  Registry  of  Vital 
Records  and  Statistics  were  used.  WRISS  cases 
that  died  in  the  ED  were  excluded  to  avoid 
double-counting.  Because  the  mortality  file  for 
1994  is  not  yet  complete,  it  should  be 
emphasized  that  figures  presented  here  are 
preliminary  estimates.  For  this  reason,  rates 
are  stated  as  ranges .  The  number  of  nonfatal 
cases  was  adjusted  for  underreporting. 

The  estimated  case  fatality  rate  (CFR)  for 
violence-related  GSWs  (excluding  BB  guns)  was 
much  higher  than  for  SIWs,  15-17%  vs  1-3%. 
Although  women  were  at  lower  risk  for  GSWs,  the 
CFR  for  females  was  higher  than  for  males,  30- 
36%  vs  13-15%;  the  CFR  was  highest  for  white 
females  (53-58%)  and  lowest  for  black  females 
(19-21%)  .  Thus,  although  white  women  had  a 
relatively  low  risk  of  GSW  injury,  they  were 
more  likely  to  die  if  injured.  The  higher  CFR 
for  white  females  requires  further 
investigation,  for  example  looking  for  possible 
differences  in  age,  circumstance,  and  victim- 
offender  relationship.  CFRs  for  males  and 
females  for  SIWs  were  similar  (1-3%). 

Youth  15-24  years  old  accounted  for  52%  of 
fatal  GSWs  and  for  62%  of  nonfatal  GSWs.  In 
contrast,  this  age  group  represented  16%  of 
fatal  SIWs  and  40%  of  nonfatal  SIWs.  Rate  ratios 
of  younger  (0-24)  to  older  (25+)  victims 
differed  for  GSWs  and  SIWs  as  well  as  for  fatal 
and  nonfatal  cases.  For  GSWs,  the  rate  ratio  was 


2.3  for  fatal  injuries  and  3.6  for  nonfatal 
injuries.  That  is,  younger  victims  were  twice  as 
likely  to  die  and  almost  four  times  as  likely  to 
be  injured.  For  SIWs,  the  rate  ratio  was  0.3  for 
fatal  injuries  and  1.4  for  nonfatal  injuries. 
That  is,  younger  victims  were  less  likely  to  die 
but  slightly  more  likely  to  be  injured.  Although 
black  residents  were  overrepresented  as  victims, 
especially  for  GSWs,  the  CFR  was  higher  for 
white  GSW  victims  (25%  vs  15%) .  There  was  no 
difference  in  CFRs  by  race/ethnicity  for  SIWs. 
These  differences  in  CFRs  suggest  the  need  for 
additional  research. 

Conclusions 

Statewide  surveillance  of  weapon-related 
injuries  is  feasible  and  can  provide  important 
new  insights  on  the  epidemiology  of  these 
injuries.  With  the  first  year  of  statewide  WRISS 
data,  we  are  beginning  to  uncover  important 
differences  between  fatal  and  nonfatal  injury 
rates  and  differences  between  communities.  In 
some  areas- -for  example,  victim-offender 
relationship  and  circumstance- -surveillance  data 
are  suggestive,  but  because  of  high  levels  of 
missing  data,  are  difficult  to  interpret.  Basic 
surveillance  may  need  to  be  supplemented  in 
these  areas  by  more  targeted,  in-depth  studies. 
On  the  other  hand,  surveillance  data  are  very 
useful  for  identifying  community-specific  risks. 

WRISS  is  embarking  on  a  new  phase  which 
includes  linkage  with  additional  data  sets, 
including  hospital  discharge  and  mortality  data. 
Through  data  linkage  and  statewide  surveillance, 
we  hope  to  be  able  to  provide  additional 
information  about  the  cost  and  potential  long- 
term  medical  effects  of  weapon-related  injuries. 
Data  over  time  also  will  be  useful  for  the 
evaluation  of  community-specific  interventions. 
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Philadelphia  is  the  fifth  largest  city  in 
the  United  States  and  has  an  estimated 
population  of  approximately  1.6  million  people. 
Philadelphia  has  been  impacted  by  violence  in 
many  ways.  The  residents  have  all  suffered 
because  of  this  problem.  Therefore,  the  city 
of  "brotherly  love"  created  a  community-based, 
public/private,  peace  promotion/  violence 
reduction,  initiative  known  as  Operation  Peace 
in  Philadelphia  or  O.P.P  designed  to  address 
this  issue. 

The  vision  for  this  initiative  is  that  by 
the  year  2000,  there  will  be  a  significant 
reduction  in  violent  behaviors,  preventable 
deaths  and  disabilities,  and  an  enhanced  quality 
of  life  for  all  residents.  The  mission  is  to 
establish  new  ideals,  beliefs  and  lifestyles  for 
all  Philadelphians  as  it  relates  to 
interpersonal  violence. 

The  initiative  consists  of  three 
components:  1.  a  community  collaboration  made  of 
action  teams  charged  with  development, 
and  evaluation  of  prevention 
a  long-term  media/awareness 
3.  an  innovative  information 
This  third  component  is  to  help 
enhance  research  and  evaluation.  It  encompasses 
the  Philadelphia  Interdisciplinary  Youth 
Fatality  Review  Team  (PIYFRT) . 

The  Philadelphia  Interdisciplinary  Youth 
Fatality  Review  Team  represents  a  comprehensive 
effort  to  enhance  the  research  of  interpersonal 
violence  involving  youth.  The  PIYFRT  may  be  the 
only  death  review  process  in  the  nation  which  is 
also  a  major  component  of  a  city's  violence 
prevention  initiative.  It  may  also  be  the  first 
child  death  review  team  in  the  nation  with  a 
subcommittee  focusing  on  youth  homicides. 

The  mission  is  to  prevent  future  child 
deaths  by  review,  analysis  and  subsequent 
initiation  of  corrective  action.  The  PIYFRT  is 
modelled  after  the  child  death  review  process 
(1) .  Child  Death  Review  Teams  are  routine, 
systematic,  multi-agency,  multi-disciplinary 
processes.  They  begin  collecting  and  organizing 
diverse  data,  in  order  to  develop  policies  that 
will  have  measurable  impact  on  the  problem  of 
youth  fatality.  The  PIYFRT  also  focuses  on 
unintentional  and  natural  deaths  of  youth. 

The  members  of  Philadelphia's  youth 
homicide  subcommittee  include  representatives 
from  the  Health  Department,  the  Office  of  the 
Medical  Examiner  and  its  Grief  Assistance 
Program,  the  police  and  fire  departments,  the 
District  Attorney's  Office,   Family  Court,   the 
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Department  of  Human  Services,  the  school 
district,  two  major  community-based 
organizations:  the  Philadelphia  Anti-Drug  Anti- 
Violence  Network  and  Congresso  de  Latinos,  and 
local  hospitals.  This  collaboration  recognizes 
that  no  one  agency  or  organization  has  the 
capacity  to  chronicle  all  the  circumstances 
surrounding  the  lives  of  those  involved  in  youth 
fatalities . 


The  subcommittee  meets  once  a  month  to 
review  homicides  of  persons  21  and  under.  The 
committee  members  share  information  from  their 
respective  agencies  in  order  to  complete  the 
data  form  (Appendix  A  &  B) .  The  database 
includes  variables  depicting  the  victims, 
offenders,  and  their  families,  thus  permitting  a 
clearer  understanding  as  to  how  and  why  the 
deaths  occurred.  After  consideration  of  the 
data,  the  determination  is  made  as  to  which 
deaths  are  "preventable"  and  subsequently  what 
policy  strategies  should  be  implemented  or 
enforced  to  prevent  its  reoccurrence . 

A  "preventable"  death  is  one  in  which, 
with  retrospective  analysis,  it  is  determined 
that  with  a  reasonable  intervention  the  death 
might  have  been  prevented.  Reasonable  is 
defined  as  taking  into  consideration  the 
condition,  circumstances  or  resources  available. 

In  1994,  Philadelphia  had  113  homicides  to 
youth  0-21  years  of  age.  Homicides  represented 
2  0%  of  all  deaths  to  youth  in  this  age  category. 
Of  these  homicides,  79.6%  were  to  African- 
American  males  between  the  ag°s  of  16-21  years. 

(figure  1)  Seasonal  patterns  were  not  evident 
although  August  had  the  lowest  number  of  deaths. 
Time  of  the  injuries  was  unknown  for  32%,  but 

(71)  63%  occurred  between  3  P.M.  and  6:59  A.M. 
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Preliminary  in  depth  analysis  had  only 
been  completed  on  62  deaths  occurring  between 
January  1--June  30,  1994.  Fifty-three  percent 
of  these  homicides  occurred  on  the  streets.  For 
56.5%,  the  relationship  between  the  victim  and 
offender  was  unknown  (figure  2)  .  Known 
offenders  had  the  same  racial/ethnic 
classification  as  their  victim's,  14.5% 
acquaintances  and  4.8%  friends.  Circumstances 
for  the  deaths  were  unknown  for  40.7%.  The 
second  leading  cause  was  arguments  25.4%. 
Firearms  (77.4%)  were  the  leading  weapons 
(figure  3)  . 

Of  the  62  deaths  only  37.1%  were 
determined  to  be  "preventable"  while  16.1  %  were 
not  "preventable."  For  46.8%  (29)  it  was 
unknown  whether  the  death  could  have  been 
prevented.  The  leading  cause  for  unknown 
"Preventability"  was  missing  information  (figure 
4)  . 
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Future  analysis  will  yield: 

-type  and  number  of  agencies  with 

prior  knowledge  of  victims  and 

perpetrators . 
-age  of  victim  at  contact  with 

agencies . 
-school  problems,  etc.,  etc. 

Youth  in  Philadelphia  appear  to  be 
susceptible  to  many  of  the  same  risk  factors  for 
interpersonal  violence  highlighted  in  the 
literature  (2,3,4,5).  However  there  remains  many 
unknown  variables.  Collecting  and  examining  data 
on  youth  homicide  victims  with  an 
interdisciplinary  youth  fatality  review  team 
will  obtain  more  information  than  with  death 
certificates  or  single  agency  reporting  alone. 
This  type  of  database  will  highlight  risk 
factors  many  young  victims,  perpetrators  and 
their  families'  experience  which  may  make  them 
more  susceptible  to  interpersonal  violence. 

Our  recommendations  thus  far  include: 
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1.  Age  groupings  should  be  done  with 
caution.  Combining  15-19  year  old  youth  (for 
other  than  convenience)  may  be  nothing  more  than 
implicating  younger  youth  for  older  youth 
behavior.  A  15  year  old's  psychological  and 
physical  development  is  different  from  that  of  a 
18  or  19  year  old.  In  the  United  States,  18  year 
olds  are  considered  to  be  adults  and  therefore 
may  exhibit  adult  behavior,  i.e.,  interpersonal 
violence . 

2.  When  determining  the  relationship 
between  victim  and  perpetrator,  there  is  a  need 
to  clearly  define  a  friend  from  an  acquaintance. 

For  many  of  our  youth  there  is  a  distinct 
difference  between  a  friend  and  an  acquaintance. 
This  distinction  has  an  impact  on  behavior. 
Therefore  this  type  of  information  will  benefit 
and  augment  prevention  strategies,  e.g., 
conflict  resolution  curricula. 
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3 .  Policies  should  be  enforced  to  ensure 
that  "missing  information"  is  not  accepted  on 
the  local  level.  This  process  has  revealed  that 
even  with  multiple  agency  input  it  is  difficult 
to  obtain  all  the  information  needed  to  fully 
understand  the  problem..  This  has  far-reaching 
implications  for  state  and  national  level 
reporting,  specifically  highlighting  causality 
or  risk  factor  identification. 

Undertaking  this  routine  systematic  and 
inexpensive  research  process  allows  the 
expansion  of  knowledge  to  determine 
"preventability" .  "Preventabilitya"  with 
subsequent  policy  development,  will  assist 
communities  to  implement  necessary  changes  that 
will  ultimately  prevent  youth  from  becoming 
victims  of   homicides. 
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PHIIADKIPHU  IvrKRDIM  IPI.INARY  YolTH  FaTAIJT>  Rt  "A  1KW  TKAV1  HoMK  IM.  SlB-CoMMITI  KF. 
Decedent  Background 


3  Name: 
7  Address: 

II  Ethnicity 


14  Cause  of  dealh 


4  Age 

8.  City: 

IV  Gender:    a.  male     b    female 

15  Address  of  F.venl 


l.Ceit«: 

2.  MEO  »: 
6  DOD: 

5  DOB: 

9  Stale 

10  Zip  : 

16    Date  of  event: 


1 7    Approx.  event  lime: 


a.m.  pm 


18    Death  certificate  adequate''  1    ves   2.  no     3.  unknown  19    Problem  »(th  Death  Certificate '_ 

20   Certifier.:  l.MD     2  Mt  21    Autopsy7     lyes     2  no  22.  Where  employed?  (it  applicable) 


Report  from  MEO: 

23    Body  part  affected:  1    head       2  trunk     3  extremities 

25   Toxicology  investigation?  l.yes     2.  no 


24    Prior  weapon-related  injuries?  1.  yes     2.  no 
26.  Significant  tox  findings?    1.  alcohol     2  drugs 


3   none 


Report  from  P.D.  « 


27  Homicide  random''  I.  yes   2   no  3  unk. 

28  Perpetrotor/susp.  perpetrator  identified'' 
1    ves  2.  no  3  unknown 

29  Number  of  perpetrators: 


30  Circumstances  of  homicide 

1    argument  2   gang  activity 

3    felony  (other  crime)  4   self  defense 


5   drug  related 

6.  domestic  violence 

7.  child  abuse 

8   arson 

9.  playing  w  guns 

1 0    retaliation  Tor  event 

11    reverse  felonv 

12.  other 

13   crossfire 

99  unknown 

31.  Method  used 

Firearm 

Non- fire  arm 

1    handgun  automatic 

7   knife 

2    non-auto  revolver 

8.  fists 

3.  shotgun 

9   poison 

4  nrle 

0   other 

5   other 

6    unknown 

32.  Place  of  event 

1    high  way /street 

2.  own  residence 

3    other  residence 

4.  school  property 

5    victim's  workplace 

6.  bar'chib/tavcm 

7    recreation  area 

8.    other 

9    unknown 

10   vehicle 

33  Was  parent  parent  sub  identified  as  perp? 

1    ves  2   no         3  unknown 

34  Did  the  perpetrator  ever  have 
responsibility  for  care  or  supervision  of 

the  child  (e.g.  baby-sitter  or  teacher9) 
1    yes  2   no 

3,  unable  to  determine 

35  Relationship  of  perpetrator  to  the  vieum9 

I  parem/guardian  2.  parent  paramour 

3  spouse,  paramour 

4  ex-spouse,  ex-boylxicnd/girlfriend 

5.  grandparent  6   aunt  or  uncle 

7   fnend  8   acquaintance 

9   baby-s.tler  I0.  stranger 

II  self  12   other 

13    unknown  14   other  relative 


Report  from  DHS 

36  Number  of  siblings  living  in  home  of 
deceased  child? 

37  Deceased  has  living  children9 

1.  yes  2.  no  3.  unknown 

38.  Was  DHS  notified  of  death9 

1.  yes  2.  no  3.  unknown 

39.  Who  was  legally  responsible  for  the  care  of 
the  deceased  child? 

1    Mother  2    Father 

3   Grandparent  4.  Aunt  or  I  "ncle 

5    Foster  parent  6  Other 

40.  Who  was  primary  caretaker9 

1    Mother  2    Father 

3    Grandparent  4  Aunt  or  I  ncle 

5.  Foster  parent  6.  Parent's  paramour 

7    Parent's  neighbor      8.  Other 

41    Is  a  parent  in  prison''  1    yes  2    no 

42.  Was  victim  in  contact  with  a  s  .s.a.? 
1  yes  2.  no 

43    Which  agency  ? 

1.  food  stamps/ Medicaid 

2.  counseling  agencies 
3   oilier  agency 

4.  none 

44.  Age  when  first  in  contact  with  s.s.a.7 
45    Which  CBO  in  contact  with9 


46  Was  there  a  DHS  investigation  of  this  death9 
1.  yes  2.  no  3  unknown 

47  DHS  action  resulted: 

1.  case  substantiated    2.  not  substantiated 
3  undetermmedV  unk  4.  other  -  explain 

48  DHS  investigation  of  surviving  children1' 
1.  yes  2.  no  3  unknown 

49  Court  involved  with  siblings  or  other  family 
members  as  a  result  of  tins  death? 

1.  yes  2.  no  3  unknown 


50    Has  victim  had  relationship  with  grandparent9 
1.  Yes  2.  No 


51.  Victim  (V) /Family  history  (F): 
(choose  any  of  the  following, 

if  applicable) 

1    diagnosis  or  treatment  for  V 

severe  mental  health  problems 

2.  victimization  m  domestic  violence    V 

3   perpetrating  domestic  violence         V 


4.  vict  of  child  abuse/neglect  —  phys.  V 

5  victim  of  child  abuse  — sexual  V 

6  prior  reports  to  DHS  for  child  V 
abuse/neglect 

7  perpetrating  child  abuse  neglect       V 

8  substance  abuse  that  interferes  V 
with  ability  to  perform  daily  tasks 

9  prior  arrests/disposrtion&'  V 
convictions  for  any  crimes 

10.  contributing  medical  problems 

1 1  school  problems 

12  employment  problems 
13.  CBO  contact 

14  homeless  -  currently 

1 5  homeless  -  ever 

16  frequent  moves 

17  lost  someone  in  violent  death 

School  history 

52  School  history -circle  all  that  apply 

I  truant  2   poor  grades 

3  learning  disabled  4.  behind  in  grade 
5  change  schools  6.  discipline  prob. 
7   dropout  8.  expelled 

9.  ADHD  10.  special  ed 

II  no  problems     12.  unknown 


53  Present  grade  in  school 

54  Age  of  dropout 


V 

F 

V 

F 

V 

F 

V 

F 

V 

F 

V 

F 

V 

F 

V 

F 

LAP  Information 

55  Religious  affiliation: 

56  Were  there  kids  who  witnessed  the  homicide'' 
1    yes  2   no  3  unknown 

57  Which  services  will  be  provided? 


58    Was  victim  a  witness  to  prior  crime  event? 
lyes         2  no         3   unknown 
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Court  Information    Victim 

59  \|  tune  of  event  ■  cunent  legal  status  with 
justice  system'* 

1  bench  wan*    2.  on  probation    3,  arrest  warr 
4  completed    5   none 

60  Involved  in  drug  trafficking? 

1    ves  2.  no  3  unknown 


6 1    Juvenile  record: 
Age         Charge 


Outcome 


Perpetrator  Information: 

65.  Name: 


66    Address: 


67  City: 


69   Zip  code: 

71  Ethnicity: 

72  Race 


68  State: 
70.  Age 


P      F 


73  Gender      1  male     2.  female 

74.  Perpetrator  (P)  Tamiry  history  (F): 
(choose  any  of  the  following,  if  applicable) 

1.  diagnosis  or  treatment  for 
severe  mental  health  problems 

2.  victimization  in  domestic  violence 
3   perpetrating  domestic  violence 

4.  victim  of  child  abuse  neglect  - 
physical 

5.  victim  of  child  abuse  —  sexual 

6.  prior  reports  to  DHS  for  child 
abuse/neglect 

7   perpetrating  child  abuse/neglect 


8   substance  abuse  that  interferes  P 
with  ability  to  perform  daily  tasks 

9.  prior  arrests/'dispositionS/'convictionsP 

10.  contributing  medical  problems  P 

1 1 .  school  problems  P 

12.  employment  problems  P 

13  CBO  contact  P 

1 4  homeless  -  currently  P 

15  homeless -ever  P 


62    Placed  in  youth  residential  facility? 
Name;  Age  Length  of  Placement 


64   Placed  in  adult  correctional  facility? 
Name  Age  U-ngth  of  Placement 


63.  Adult  record 
Age        Charge 


16.  frequent  moves  P     F 

1 7  lost  someone  in  violent  death  P     F 

75.  Parent  m  prison? 
1.  yes  2.  no  3.  unknown 

School  history 

76  School  history  -  circle  all  that  apply 
1.  truant  2   poor  grades 

3.  learning  disabled  4  behind  m  grade 
5.  change  schools  6  discipline  prob 
7  dropout  8   expelled 

9.  ADHD  10  special  ed. 

1 1    no  problems     1 2   unknown 

77  Present  grade  in  school 


84  Juvenile  record: 
Age        Charge 


Outcome' 


85    Placed  in  youth  residential  facility? 

Name:  Age  Length  of  Placement 


78.  Age  of  dropout 


Court  Information 

79    Was  case  presented  to  the  prosecutor? 
1.  yes  2.  no  3  unknown 

80.  Criminal  charges  pursued  by  prosecutor?' 
1.  yes  2.  no  3.  unknown 

8 1 .  Outcome  of  court  activities,  (if  known)? 


82.  Involved  in  drug  trafficking7 

1.  yes  2.  no  3.  unknown 

83.  At  time  of  event  -  current  legal  status  with 
justice  system? 

1.  bench  warr.    2.  on  probation    3.  arrest  warr. 
4.  completed   5   none 


86.  Adult  record: 
Age        Charge 


Outcome 


87   Placed  m  adult  correctional  facility? 
Name:  Age  Length  of  Placement 


Mental  Health  Assessment  1    yes     2   no 


89   Death  investigation  adequate'' 


91    If  no,  missing  elements  are? 


1.  ves 


2  no 


3  unknown 


92   Tracking  pending  information: 

1 .  autopsy  report         2.  toxicology  report 


90   Is  this  death  preventable?  lyes     2.  no 
3  unknown  (incomplete  information) 
4.  unknown  (unable  to  decide) 


3  school  reports 
5  medical  records 
7  DHS  information 
9  MH/MR  history 
11  Other 


4.  district  attorney 
6  AID  records 
8  CBO  history 
10  Substance  Abuse 
treatment  records 


93   Prevention  strategies/policy  implications 


94  Comments 
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VIOLENCE  IN  THE  WORKPLACE:  NUMBERS,  RATES,  RISK  FACTORS, 
AND  PREVENTION  STRATEGIES 

E.  Lynn  Jenkins 
National  Institute  on  Occupational  Safety  and  Health 


Paper  not  available  for  publication 
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DEMAND  FOR  GENERAL  PRACTITIONER  SERVICES  -  A  PRACTICAL  TEST  OF  SYNTHESISED  ESTIMATION 

John  Charlton,  Office  of  Population  Censuses  and  Surveys,  London,  WC2B  6JP,  OK. 

Patrick  Heady,  Gerry  Nicolaas 


Abstract 

Synthesised   estimation    techniques   are 
methods   for  obtaining  local    estimates  by 
combining  data    which   are  not   available  for 
local    areas    with    other  data    which    are.    For 
example,    we  may  wish    to  produce  estimates 
combining  national    survey  data    with   electoral 
ward  data   available  from  a    census.    Usually 
the   opportunity   to  evaluate    the  performance 
of  such    estimates    does   not   arise,    but    owing 
to    the   existance    of  a    unique   dataset    OPCS  has 
been   able    to  evaluate  estimates   for   two  dif- 
ferent measures   at    two  different   area   levels 
-  populations   of  200,000  and  500   respec- 
tively.   The   estimates   of  serious   illness   for 
the  larger  areas   correlate   well    with  mortal- 
ity rates   for   the   same  areas,   providing  some 
evidence  of   their  validity.    A  simulation 
exercise   was   undertaken   for   the   smaller  areas 
to  validate   estimates   against   actual   rates. 
Although    the   estimates   are   not    accurate   for 
individual    areas,    they   represent    a    substan- 
tial   improvement   over  assumptions   of  equal 
consultation   rates    which    would  otherwise  have 
to  be  made.    The  same  methods   could  be   used   to 
obtain    other   estimates    of  relevance    to    the 
health    services . 


1 .  Introduction 

Synthesised  estimation  techniques  are 
methods  for  obtaining  local  estimates  by 
combining  data  which  are  not  available  for 
local  areas  with  other  data  which  are.  For 
example,  we  may  wish  to  produce  estimates 
combining  national  survey  data  with  electoral 
ward  data  available  from  a  census.  National 
survey  data  typically  provide  unreliable 
direct  estimates  for  areas  smaller  than 
fairly  broad  regions.  Synthesised  estimates 
use  the  whole  national  data  set  to  produce 
estimates  of  how  the  variable  of  interest, 
for  example  health  status,  relates  to  other 
characteristics  of  the  individuals  in  the 
sample  (such  as  age,  sex,  housing  tenure  and 
ethnic  group)  and  to  characteristics  of  the 
areas  in  which  they  live  (e.g.  rural/urban 
neighbourhoods) .  A  mathematical  model  is 
developed  that  is  used  to  'spread'  the  infor- 
mation about  the  variable  of  interest  to 
local  areas  throughout  the  country,  producing 
model-based  estimates  of  the  values  to  be 
expected.  Anyone  applying  survey  results  to 
local  situations  is  in  effect  producing  some 
form  of  synthesised  estimates,  although  they 
may  not  be  aware  of  the  technical  term.  There 
has  recently  been  an  increased  interest  in 


small  area  estimates  as  a  result  of  the  need 
for  more  careful  planning,  and  new  tech- 
niques, many  of  which  draw  upon  nested  multi- 
level modelling  have  been  developed1.  Little 
work  has  been  done,  however,  on  assessing  how 
well  the  estimates  perform  when  compared  with 
reality.  In  this  paper  we  have  taken  the 
opportunity  to  conduct  practical  tests  on  the 
performance  of  the  estimates,  using  a  par- 
ticularly useful  large  data  set. 

The  study  had  two  parts  designed  to  ad- 
dress two  different  questions  regarding  the 
reliability  of  synthesised  estimates. 

Study  (a)  produced  estimates  of  the 
prevalence  of  serious  illness  for  some  250 
medium  sized  areas  of  England  and  Wales 
(around  200,000  population  each),  and  com- 
pared these  estimates  with  mortality  in  a 
slightly  wider  age  group. 

Study  (b)  was  a  simulation  exercise  to 
investigate  how  accurate  synthesised  esti- 
mates are  at  the  really  small  area  scale,  and 
whether  model-based  confidence  intervals 
could  be  relied  on.  It  was  based  on  areas  of 
around  500  population  (200  households) ,  and 
employed  a  simple  model  with  simplifying 
model  assumptions.  Thus  in  many  respects  it 
would  resemble  a  'worst  case' .  The  modelling 
was  based  on  a  sub-sample  of  the  MSGP4  data 
to  achieve  a  sample  size  more  typical  of  most 
sample  surveys. 


2 .  Data  Sources 

2.1  The  4th  National  Study  of  Morbidity 
in  General  Practice,  1991-92  (MSGP4) . 

In  the  United  Kingdom  there  is  a  system 
of  universal  patient  registration  with  GPs, 
and  nearly  all  residents  have  their  health 
care  managed  via  the  practice  with  which  they 
are  registered.  Apart  from  medical  emergen- 
cies, secondary  care  is  achieved  through 
referral  by  general  practitioners.  Following 
discharge,  relevant  morbidity  data  are  for- 
warded from  hospital  to  practices.  Thus  the 
general  practice  provides  the  ideal  setting 
for  measuring  health  status  as  encountered  by 
the  medical  profession.  Corresponding  denomi- 
nators are  provided  from  the  practice  age/sex 
registers.  The  survey  data  in  our  study  comes 
from  MSGP42,  a  study  which  collected  data  on 
all  face  to  face  contacts  between  502,493 
patients  and  their  general  practitioners 
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throughout  England  and  Wales  between  1  Sep- 
tember 1991  and  31  August  1992  -  providing 
data  on  468,042  patient-years  at  risk.   The 
60  practices  taking  part  in  the  study  were 
not  randomly  selected,  since  they  had  to 
volunteer  to  take  part  in  the  survey,  and 
have  the  necessary  computer  systems  to  col- 
lect the  information  required.  They  were 
selected  from  volunteers  to  achieve  the  best 
possible  national  coverage,  including  prac- 
tices from  all  Regional  Health  Authorities 
and  with  diverse  socio-economic  characteris- 
tics. Doctors  recorded  their  diagnoses  on 
computer  using  the  Read  coding  thesaurus 
which  is  in  general  use  in  the  UK.  There 
could  be  any  number  of  diagnoses  per  contact, 
and  these  were  subsequently  mapped  to  ICD9  by 
OPCS.  In  addition  a  linked  interview  survey 
was  conducted  to  collect  socio-economic  de- 
tails concerning  the  patients.  This  survey 
achieved  8  3  per  cent  coverage.  The  socio- 
economic data  were  similar  in  distribution  to 
that  recorded  for  England  and  Wales  in  the 
1991  Census. The  data  thus  enabled  prevalence, 
incidence,  and  health  utilisation  measures  to 
be  calculated  for  different  socio-economic 
groups,  and  provided  near  saturation  coverage 
of  the  immediate  locality  of  the  practice  in 
about  half  of  the  general  practices  taking 
part  in  the  study,  thus  making  it  possible  to 
know  the  true  consultation  rates  for  a  number 
of  small  areas. 

2.2  The  Two  per  cent  Sample  of  Anonymised 
individual  census  Records  from  the  1991  cen- 
sus (SAR) . 

The  1991  census  sample  of  anonymised 
records  is  described  fully  elsewhere3,  and 
was  the  source  of  most  of  the  local  data  used 
to  spread  the  survey  results  to  medium  sized 
areas  of  England  and  Wales  (study  a) .  It 
provided  precise  information  for  all  the 
socio-economic  items  apart  from  smoking 
prevalence,  which  was  estimated  from  General 
Household  Survey4  figures  for  the  region  in 
which  the  medium  sized  area  was  situated. 
There  are  253  SAR-areas  identifiable,  corre- 
sponding to  local  authorities,  or  in  the  case 
of  very  small  authorities,  to  two  or  more 
adjacent  authorities  combined.  The  sample  for 
each  area  was  approximately  4,000,  out  of  a 
population  of  around  200,000. 


3.  Study  (a) :   Estimates  for  medium  size 
areas,  with  cross-validation  against  inde- 
pendent data . 


ity.  Because  different  models  might  be  re- 
quired for  different  age/sex  groups,  separate 
models  were  fitted  for  males  and  females 
the  age  groups  0-15,  16-44,  45-64  and  65  and 
over.  In  this  paper  we  concentrate  only  on 
the  model  for  men  aged  16-44.  The  other  re- 
sults, and  a  fuller  explanation  of  the  model- 
ling, can  be  found  in  the  study  repoi t  . 

The  probability  of  consulting  for  a  seri- 
ous illness  was  treated  as  a  binomial  vari- 
able and  modelled  using  GLIM4.  The  following 
socioeconomic  variables  were  analysed  in 
relation  to  the  variable  of  interest:  age 
(single  years) ;  ethnic  group;  housing  tenure; 
social  class;  economic  position  (eg  whether 
unemployed) ;  whether  in  practice  for  full 
year;  family  type  (combining  marital/cohabit- 
ing status  with  having  children) ;  smoking 
status;  whether  living  in  a  rural  or  urban 
locality  (Department  of  Environment  classifi- 
cation of  enumeration  districts) ;  crow-fly 
distance  from  practice;  and  general  level  of 
limiting  long-term  illness  in  the  locality  of 
residence  (1991  Census) .  The  number  of  days 
that  a  patient  was  at  risk  of  consulting  was 
explicitly  included  in  the  model  as  an  'off- 
set' . 

The  model  was  a  multi-level  one,  with 
practice  as  a  first  level  and  patient  as  the 
second  level.  Practice  variations  were  ex- 
plicitly allowed  for  by  fitting  a  separate 
practice  constant  for  each  practice  (as  a 
'fixed  effect' ,  since  the  practices  were  not 
a  random  sample) .  This  allowed  for  the  possi- 
bility that  each  practice  would  have  its  own 
level  of  recorded  morbidity,  over  and  above 
that  which  was  due  to  patient  characteris- 
tics. In  the  event  the  between-practice  vari- 
ations proved  to  be  greater  than  the  varia- 
tions due  to  socioeconomic  characteristics. 

3.2  Results 

The  resultant  multivariate  logistic  re- 
gression model  provided  an  estimate  of  the 
probability  of  consulting  for  serious  illness 
for  individuals  with  any  combination  of  the 
characteristics  used  in  the  model.  In  order 
to  calculate  the  estimated  consultation  rates 
for  each  SAR-area,  we  used  the  parameter 
estimates  derived  from  this  model  to  calcu- 
late individual  consultation  probabilities 
for  each  of  the  4,000  people  in  the  SAR- 
sample  for  each  area.  The  synthethised  esti- 
mate of  the  area's  prevalence  was  then  the 
mean  of  these  rates. 


3.1  Methods 

The  variable  of  interest  was  the  propor- 
tion of  people  expected  to  consult  for  seri- 
ous illness,  ie  any  condition  that  was  life 
threatening,  requiring  major  surgery  or  in- 
tensive care,  or  having  the  potential  for 
serious  complications  or  recurring  disabil- 


The  map  (Figure  1)  shows  how  the  esti- 
mated probability  of  consulting  for  serious 
illness  by  men  aged  16-44  varied  between 
different  parts  of  the  country.  The  pattern 
was  much  as  would  be  expected.  In  order  to 
cross  validate  the  results  these  estimates 
were  compared  with  an  independent  measure, 
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age  standardised  mortality  ratios  for  ages 
15-64. 


Figure   1:    Synthesised   estimates   of   the 
percentage   of  males   aged   16-44   consulting 
their  general  practitioner  for  serious 
illness,    based  on    the  MSCP4   survey  and   the   2 
per  cent   sample   of  anonymised  individual 
records   from    the   1991    census. 
Local   Authority  areas   of  England  and  Wales. 


Males 

Key  percentage  of  consuming 


It  had  been  hypothesised  that  serious 
illness  in  ages  16-44  should  be  correlated 
with  mortality  in  the  somewhat  wider  age  band 
15-64.  Figure  2  shows  the  relationship  be- 
tween the  two  measures,  which  was  linear, 
with  a  correlation  coefficient  of  0.81.  This 
provides  some  reassurance  that  the  estimates 
of  serious  illness  rates  are  valid  ones.  An 
independent  analysis,  for  females  in  this  age 
group,  gave  similar  results2. 


Figure   2  -.Comparison   of   the   estimated  proportion 
of  men   aged   16-64   consulting  for   serious 
illnes,    with    SMRs    for  men   aged   15-64,    by  Local 
Authority  area 
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4 .  Study  (b)  Estimating  consultation 
rates  for  very  small  areas  -  confidence  in- 
tervals . 

4.1  Methods 

Enumeration  districts  (EDs)  were  chosen 
as  the  areas  for  which  estimates  were  to  be 
made.  These  are  the  smallest  areas  for  which 
OPCS  provides  information  about  population 
characteristics.  They  consist  of  approxi- 
mately 500  individuals,  or  200  households. 
The  aim  was  to  estimate  how  consultation 
rates  (for  any  reason)  differed  from  the 
average  for  the  practice  as  a  whole. 

To  approximate  the  synthesised  estimation 
process  as  it  would  apply  for  a  sample  sur- 
vey, the  modelling  was  based  only  on  a  sub- 
sample  of  the  MSGP4  data.  In  addition,  the 
dataset  analysed  was  restricted  to  those  EDs 
for  which  the  MSGP4  achieved  70  per  cent 
coverage  or  better.  This  dataset  comprised 
228  EDs  in  38  of  the  original  60  practices 
(for  9  of  the  38  practices  the  dataset  con- 
tained only  1  ED,  for  the  remaining  29  there 
were  2  EDs  or  more) .  The  reason  for  restrict- 
ing the  data  to  these  EDs  was  so  that  we 
could  know  the  true  consultation  rates,  and 
also  estimate  them  for  comparison.  For  the 
purpose  of  the  simulation  exercise  the  sam- 
pled individuals  in  the  228  EDs  were  taken  to 
be  the  true  population,  and  each  ED's  socio- 
economic characteristics  were  derived  from 
the  MSGP4  data  rather  than  from  the  census 
itself.  The  only  exceptions  to  this  were  the 
percentage  with  a  limiting  long-term  illness, 
which  was  taken  from  census  data  because  it 
was  not  covered  by  the  MSGP4  study,  and  the 
urban/rural  classification. 


The  following  variables  were  included  in 
the  analyses: 

i)   The  number  of  consultations  for  any 
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reason  was  taken  as  the  dependent  variable. 
The  objective  was  to  estimate  the  difference 
between  the  mean  consultation  rate  for  an  ED 
and  the  average  for  the  practice. 

ii)   Patient  level  data  comprised:  age 
and  sex;  single;  married  or  cohabiting;  wid- 
owed, separated  or  divorced;  White;  Indian, 
Pakistani  or  Bangladeshi;  Black  or  other; 
owner  occupier;  renting  accommodation;  work- 
ing; not  working. 

iii)   ED  level  data  (from  the  Census) 
comprised:  percentage  of  inhabitants  with  a 
limiting  long-term  illness;  urban  or  rural. 

iv)   Practice  level  data  comprised:  list 
size  of  practice;  number  of  patients  per 
principle  in  the  practice. 

The  aim  was  to  estimate  the  difference 
between  the  average  consultation  rate  for 
each  ED  and  the  rate  for  the  practice  as  a 
whole,  ie  we  modelled  the  geographical  varia- 
tions within  each  practice  locality. 

We  ran  10  simulations,  each  time  dividing 
the  practices  into  2  sets:  those  that  were 
used  to  fit  the  model  (the  model-fitting 
set) ;  and  those  that  were  used  to  test  the 
accuracy  of  the  resulting  estimates  (the 
estimate-testing  set) .  In  each  simulation 
practices  were  chosen  with  probability  0.5 
for  the  model-fitting  set  (resulting  in  an 
average  of  19  practices) .  The  remaining  19  or 
so  practices  formed  the  estimate-testing  set, 
but  in  fact  only  those  practices  with  2  or 
more  EDs  were  used  -  since  the  objective  was 
to  assess  the  accuracy  with  which  we  could 
estimate  differences  between  ED  and  practice 
means.  For  this  reason  the  effective  esti- 
mate-testing set  contained  on  average  about 
109  EDs  from  14  practices,  although  due  to 
differencing  from  the  practice  means  this 
resulted  in  only  95  or  so  independent   point- 
estimates. 


of  variances  fitted  by  ML3  and  not  the 
variances  of  ML3's  own  parameter  estimates. 
Similarily  we  have  assumed  normality  where  it 
is  arguable  that  Poisson  or  Hypergeometric 
distributions  would  have  been  more  appropri- 
ate. The  reasons  for  sticking  with  simplicity 
were  twofold  -  firstly  that  it  was  easier  to 
implement.  Secondly  that  if  the  test  performs 
reasonably  well  with  simple  methods,  then 
more  sophisticated  techniques  would  perform 
at  least  as  well. 

A  final  essential  point  to  note  is  that 
in  our  procedures  the  data  used  to  evaluate 
the  estimates  is  almost  entirely  independent 
of  the  data  used  to  derive  the  model  param- 
eters. The  only  element  of  dependence  is  that 
the  choice  of  variables  to  include  in  the 
model  was  made  on  the  basis  of  an  initial 
regression  analysis  of  data  from  all  38  prac- 
tices in  228  EDs.  The  exercise  thus  provides 
an  empirical  test  of  the  validity  of  the 
estimates  which  is  almost  entirely  free  from 
a  priori  assumptions . 


4.2  Results 

Table  1  shows  the  correlations  between 
the  true  and  estimated  ED  values  for  each 
simulation,  ie  ED  differences  from  the  prac- 
tice mean.  The  values  are  consistent,  ranging 
from  a  minimum  of  0.58  to  a  maximum  of  0.71 
with  a  mean  of  0.67.  Figure  3  shows  the  scat- 
ter plot  for  the  second  simulation  (whose 
correlation  coefficient  is  also  0.67).  It 
shows  that  the  relationship  between  estimates 
and  actual  values  is  linear. 

Table   1:    Correlation  between   observed  and 
predicted  difference  between   ED  average   and 
practice  average   consultation   rates. 

Simulation     Correlation  coefficient 


In  order  to  fit  the  model,  a  subsample  of 
1  person  in  8  was  drawn  from  the  model-fit- 
ting set,  giving  a  sample  of  around  5,500  - 
similar  to  an  ordinary  survey  size  with  19 
primary  sampling  units.  The  model  was  fitted 
using  the  ML3  6  modelling  program,  using  a 
generalised  least  squares  method  7,  and  was  a 
3-level  model  (person,  ED,  practice) ,  with 
fixed  linear  regression  effects,  and  separate 
variance  estimates  for  person,  ED,  and  prac- 
tice levels.  The  estimated  variances  of  the 
random  effects  were  combined  to  produce  con- 
fidence intervals  for  the  EDs,  using  the 
between-ED  and  the  between-person  variances 
from  the  multi-level  model.  This  was  done  for 
the  first  7  simulations  only,  due  to  time 
constraints.  The  proportion  of  intervals 
which  contained  the  true  value  was  noted. 

Our  approach  was  statistically  fairly 
simplistic,  for  example  we  only  took  account 


1 

0.71 

2 

0.67 

3 

0.71 

4 

0.68 

5 

0.58 

6 

0.66 

7 

0.63 

8 

0.69 

9 

0.70 

10 

0.65 

Mean  correlation:  0.67 

Table  2  shows  the  performance  of  the 
confidence  intervals  for  these  estimates,  in 
terms  of  the  number  of  times  that  the  true  ED 
value  fell  outside  the  95  per  cent  confidence 
interval.  The  mean  percentage  of  values  that 
fell  outside  the  confidence  interval  (5.7)  is 
close  to  the  theoretical  target  of  5.0.  Al- 
though the  proportion  was  not  at  all  stable, 
ranging  from  0.0  to  12.0  per  cent,  the  vari- 
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Figure  3:   Difference  between  ED  consultation   rate  and  practice 
mean   consultation   rate:    observed  against   expected    (Simulation  2) 


■1.5     -1.0     -.5      0.0 
Estimated  difference 


ability  is  not  surprising.  The  number  falling 
outside  the  interval  is  likely  to  approximate 
a  Poisson  variable  with  standard  deviation 
equal  to  the  square  root  of  the  expected 
number,  which  at  about  6  is  small.  The  clus- 
tered sample,  together  with  the  mutual  de- 
pendence arising  from  differencing  from  the 
mean  within  clusters,  will  raise  the  vari- 
ability compared  with  the  simple  random  sam- 
ple case. 

Table   2.    Percentage   of  predicted  vaues   outside 
the   95%  confidence   interval. 

Simulation     Percent  outside 
interval 

1  6.0 

2  7.0 

3  0.8 

4  6.9 

5  12.0 

6  0.0 

7  7.2 

Mean  percent  outside  interval:  5.7 


5 .  Discussion 

Before  reviewing  the  technical  issues  it 
is  worth  briefly  considering  the  nature  of 
the  estimates  in  studies  (a)  and  (b) . 
Strictly  speaking  the  dependent  variable  is 
behaviour  -  patient  contact  with  their  GP, 
which  can  be  determined  by  patient  and  prac- 
tice factors.  Our  models  allow  for  both  de- 
mand (patient  characteristics)  and  supply 


(practice  characteristics,  or  dummy  practice 
variables),  but  the  estimates  of  differences 
between  areas  depend  entirely  on  the  demand 
variables  (the  effect  of  supply  has  been 
removed).  The  value  of  such  estimates  to 
planners  is  as  predictors  of  demand,  and 
hence  (often)  need.  Planners  need  to  differ- 
entiate between  smaller  areas  within  some 
macro-area  in  order  to  make  judgements  about 
where  to  allocate  resources. 

The  results  presented  above  are  encourag- 
ing because  they  show  that  it  is  possible  to 
carry  out  this  process  at  two  levels: 

distinguishing  medium  size  area  with 
about  200,000  inhabitants  within  the  context 
of  national  variation;  and 

distinguishing  EDs  with  about  500 
inhabitants  within  an  immediate  locality  (of 
a  general  practice) . 

The  fact  that  the  process  works  reason- 
ably well  for  both  examples  suggests  that  it 
would  probably  produce  useful  results  for 
areas  of  intermediate  size  (for  example 
wards,  with  populations  averaging  5,000).  The 
fact  that  the  confidence  intervals  worked 
well  on  average  is  also  encouraging. 

But  does  the  estimation  process  work  well 
enough?  The  answer  depends  on  what  one  is 
trying  to  do.  One  criterion  would  be  whether 
it  is  possible  to  place  reasonably  narrow 
confidence  intervals  around  the  estimates  for 
individual  areas.  Although  we  have  not  super- 
imposed confidence  intervals  around  the  scat- 
ter plots,  a  glance  at  the  degree  of  scatter, 
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say  for  the  ED  estimates,  would  indicate  that 
confidence  intervals  for  any  individual  areas 
are  likely  to  be  wide.  The  ED  estimates  would 
certainly  fail  by  this  criterion.  At  this 
stage  we  cannot  be  sure  about  the  width  of 
confidence  intervals  for  the  SAR-level  esti- 
mates, since  the  data  structure  did  not  per- 
mit us  to  calculate  between-SAR  variance.  But 
the  confidence  interval  criterion  may  not  be 
the  most  appropriate  in  the  context  of  health 
service  planning.  Although  the  estimate  of 
resource  demand  for  any  individual  area  may 
not  be  very  accurate,  the  synthetic  estimates 
may  still  be  valuable  if  on  average  they 
produce  a  better  estimate  of  demand  than 
would  be  obtained  by  simple  proportionate 
allocation.  The  fact  that  the  synthesised 
estimates  account  for  a  substantial  propor- 
tion of  the  variation  of  the  true  values 
means  that  they  would  achieve  this  objective. 
Synthesised  estimates  may  also  be  useful  in 
another  way  -  as  an  indication  of  the  level 
of  demand  (or  whatever)  that  would  be  ex- 
pected on  the  basis  of  the  characteristics 
that  have  been  used  to  derive  the  estimate. 
Where  alternative  data  sources  exist  relating 
to  the  actual  outcome,  these  can  be  compared 
with  the  synthesised  estimates  to  identify 
those  areas  whose  outcomes  are  notably  above 
or  below  expectations.  By  identifying  these 
areas  they  may  help  in  the  search  for  addi- 
tional factors,  possibly  relating  to  the 
local  environment  or  policies.  Thus  for  exam- 
ple general  practices  could  compare  their 
observed  consultation  rates  with  what  would 
be  expected  independent  of  practice  and  local 
environmental  factors. 

It  is  important  to  note  the  obvious,  that 
because  synthesised  estimates  are  based  on  a 
model,  good  estimates  can  only  be  produced 
when  a  good  model  has  been  developed  linking 
the  various  data.  Care  needs  to  be  taken  in 
evaluating  the  appropriateness  of  the  model 
to  be  used,  and  wherever  possible  the  esti- 
mates should  be  evaluated  against  external 
criteria  for  face  and  other  forms  of  valid- 
ity. 

Estimates  useful  to  the  health  services 
could  be  produced  from  a  wide  range  of  sur- 
veys. While  the  MSGP4  can  provide  estimates 
based  on  contact  with  general  practitioners, 
the  Health  Survey  for  England8  and  the  Gen- 
eral Household  Survey  (GHS)4  contain  informa- 


tion on  contacts  with  other  health  service 
providers,  and  also  health  related  behaviour. 
Surveys  can  be  used  to  provide  estimates  of 
morbidity  independent  of  health  service  con- 
tacts such  as  self  rating  of  health,  chronic 
illness,  acute  sickness,  psychiatric  morbid- 
ity, diet,  contraceptive  use,  infant  feeding 
etc.  While  OPCS  does  not  have  a  specific 
programme  of  work  in  this  area  -  the  work  we 
do  is  driven  by  demand  from  clients  -  we  are 
keen  to  explore  the  applicably  of  synthesised 
estimation  techniques  to  these  other  areas. 
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COLLECTION  AND  UTILIZATION  OF  CHILD  ABUSE  STATISTICS 
IN  AMERICAN  INDIAN  COMMUNITIES 

Michelle  Chino,  Albuquerque  Area  Indian  Health  Board,  Inc. 


Public  Health  Research  in  Indian  Communities 

Public  health  research  in  American  Indian 
communities  involves  many  complex  issues  that  may  both 
help  and  hinder  the  development  of  an  effective  research 
methodology  and  the  collection,  analysis,  and  utilization  of 
data.  These  issues  include:  1)  the  unique  strengths  and 
diversity  of  Indian  cultures:  2)  the  complicated 
relationships  that  exist  between  federal,  state,  and  tribal 
agencies;  3)  the  vast  distances  between  communities  and 
services  that  exist  in  rural  areas;  4)  extremely  limited 
human  and  financial  resources;  5)  overlapping  and  often 
conflicting  legal  and  jurisdictional  authorities:  and  6)  an 
array  of  social  issues  including  poverty,  substance  abuse, 
modernization,  and  assimilation.  Defining  the  parameters 
of  any  health  issue  requires  a  broad  based  understanding 
of  the  unique  social,  cultural,  and  political  dynamics  of 
Indian  tribes  and  tribal  communities. 

Research  and  data  collection  must  also  take  into 
account  a  tribe's  status  as  a  sovereign  nation.  The  concept 
of  sovereignty  gives  federally  recognized  tribes  the  right  to 
exercise  basic  governmental  powers.  In  addition,  treaties, 
recent  Indian  legislation,  and  the  US  Trust  responsibility 
mandate  the  Federal  government  to  make  provisions  for 
basic  needs  such  as  health  care,  education,  and  housing. 
One  result  of  this  unique  status  is  that,  in  any  given  tribal 
community,  there  may  be  a  combination  of  tribal,  federal, 
and  state  agencies  providing  services  and  collecting  data. 
Each  agency  may  have  different  roles,  responsibilities, 
protocols  for  obtaining  information,  and  levels  of 
sophistication  with  regard  to  data  management. 

With  few  exceptions,  the  more  than  500  Indian 
tribes  and  Alaskan  Native  villages  in  the  US  range  in  size 
from  a  few  hundred  to  a  few  thousand  people.  While  low 
population  density  does  not  preclude  the  existence  of 
serious  public  health  problems,  it  makes  it  difficult  to 
document  prevalence  and  incidence.  In  addition  most,  if 
not  all,  tribal  groups  are  in  some  stage  of  epidemiological 
and  demographic  transition,  reflected  in  part  by  the 
changing  causes  of  morbidity  and  mortality  among  Indian 
people.  This  transitional  status  supports  both  the  need  for, 
and  the  opportunity  for  defining  and  addressing  public- 
health  issues.  For  example,  cross  cultural  studies  have 
shown  that  societies  in  transition  are  especially  susceptible 
to  issues  such  as  child  abuse. 

As  federal  budgets  reduce  and  re-define  the 
services  available  to  Indian  people,  tribes  are  faced  with 
the  need  to  plan,  develop,  and  fund  their  own  prevention 
and  intervention  programs.  The  changing  structure  of  the 
current  health  care  system  demands  a  new  approach  to 
addressing  the  unique  needs  of  Indian  people  and  the 
individual  needs  of  tribal  communities. 

Child  Abuse  and  Neglect 
Child  abuse  is  being  increasingly  recognized  as  a 
public  health  issue.  This  approach  facilitates  study  of  the 
history  of  the  issue,  insight  into  the  complex  bio-social 
processes  at  work,  and  the  development  of  effective 
strategies  for  prevention  and  intervention.  A  public  health 
approach  also  recognizes  that  the  cultural,  social,  and 
political  context  of  the  community  will  determine  what 
constitutes  abuse;  situations  that  excuse  or  mitigate  abuse; 
appropriate  responses  that  can  be  applied;  and  what  the 
resources  should  be  versus  what  is  actually  available.  To 
fully  comprehend  and  evaluate  the  issues  that  accompany 


research  on  child  abuse  and  neglect  among  Indian  people, 
it  is  necessary  to  understand  the  interaction  between  health 
and  social  services,  Indian  law,  tribal  culture,  and  the 
political  systems  that  are  both  stressed,  and  called  upon, 
when  the  breakdown  of  Indian  family  relationships  result 
in  the  maltreatment  of  children. 

In  tribal  communities  the  agencies  involved  in 
responding  to  the  victim,  the  perpetrator,  and  the  families 
include  tribal,  state,  and  federal  health  care  services,  social 
services,  law  enforcement,  courts,  and  schools.  Most 
Indian  communities  also  have  a  designated  Child 
Protection  Team.  This  multidisciplinary  team  of  service 
providers  is  designed  to  monitor  and  follow-up  reported 
incidents  of  child  abuse  and  ensure  that  available  services 
are  provided  and  are  not  redundant.  Even  so,  which 
agency  has  primary  responsibility  for  reporting  and 
investigation  and  who  has  jurisdiction  is  rarely  agreed 
upon.  In  addition,  each  agency  requires  varying  types  of 
data  and  information  and  may  have  varying  levels  of 
sophistication  with  regard  to  human  and  technological 
resources.  With  the  current  and  impending  changes  in 
health  care  and  related  services,  in  order  to  develop  or  re- 
define the  existing  service  delivery  system,  tribes  need 
comprehensive  yet  simple  methods  that  will  maximize  the 
availability  and  the  utility  of  local  data. 

The  National  IHS  Child  Abuse  and 
Child  Neglect  Project 

The  Indian  Health  Service  (IHS)  is  a  federal  agency 
that  is  part  of  the  trust  relationship  between  tribes  and  the 
US  government.  The  IHS  provides  primary  and 
preventive  health  care  services  to  a  majority  of  reservation 
based  tribal  populations.  The  IHS  has  also  been  the 
primary  source  of  health  research  and  statistics  for  tribes. 

A  recent  national  level  project  (Chino  et  al.,  1993) 
to  address  the  issues  of  child  maltreatment  in  Indian 
communities  was  part  of  the  IHS  research  component. 
There  were  three  objectives  of  the  national  project:  1 )  to 
research  the  scope  and  the  long  term  effects  of  child 
neglect,  physical  and  sexual  abuse  on  Indian  children;  2) 
to  study  the  role  of  the  Indian  Health  Service  in  issues  of 
child  protection  and  child  maltreatment:  and  3)  develop  a 
model  intervention  program.  What  started  as  an 
administrative  assessment  of  policy  and  procedure 
however,  rapidly  became  a  study  of  risk  factors, 
causation,  resource  utilization,  family  and  community 
dynamics,  local  politics  and  response  systems,  and  the 
severe  gaps  that  exist  between  the  administrative  levels  and 
services  levels  of  the  health  care  system. 

Part  of  the  initial  intent  of  the  project  was  to  evaluate 
existing  data,  determine  prevalence  and  incidence  rates, 
assess  variability  between  tribes,  and  make  comparisons 
with  data  from  the  general  population.  This  information, 
for  the  most  part,  simply  did  not  exist.  When  information 
or  data  were  available  they  were  often  limited  in  scope 
(e.g.,  only  cases  of  sexual  abuse  were  counted)  and  rarely 
in  a  format  amenable  to  analysis  (e.g.,  only  aggregate 
counts). 

The  national  level  project  was  successful  in 
obtaining  caseload  information  on  over  2000  incidents  of 
child  maltreatment  from  37  tribal  communities  across  the 
country.  Although  the  data  were  not  population  based  or 
truly  representative  of  any  individual  tribe,  the  data  set 
provided  the  first  national  level  identification  of  variables 
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ot  interest  and  baseline-  estimates  their  impact.  The  process 
itself  was  invaluable  in:  understanding  the  constraints  that 
inhibit  effective  data  collection  in  tribal  communities; 
developing  comprehensive  but  simple  analytic  techniques; 
and  identifying  the  need  for  tribal  research  to  focus  on 
community  planning. 

Barriers  to  Effective  Data  ( 'olle<  Hon 
Surveillance  issues  (e.g.,  identification  and 
reporting)  had  a  major  impact  on  data  collection  and 
continue  to  be  a  problem  for  many  tribes.  Issues  include 
denial,  the  reluctance  to  recognize  abuse,  and  the  inability 
to  separate  abuse  from  poverty  or  cultural  practices.  In 
addition,  despite  federal  mandates  for  reporting,  many 
people  are  reluctant  to  report  abuse.  Sensitive  issues  such 
as  child  abuse  can  stigmatize  individuals  and  communities 
which  severely  constrains  the  collection  of  data  and 
hinders  tribal  communities  from  developing  an  aggressive, 
systematic  response. 

Data  collection  in  tribal  agencies  is  currently  guided 
In  the  needs  of  the  funding  agency,  usually  at  the  federal 
level.  While  this  ensures  at  least  some  data  is  collected  at 
the  local  level,  the  format  usually  reduces  its  utility  for 
agency  and  community  planning  and  analysis.  Even  when 
information  is  available  usage  is  constrained  by:  1)  a  lack 
of  a  systematic,  computerized  system  for  data  management 
within  and  between  agencies;  2)  varying  levels  of 
sophistication  and  ability;  3)  reliance  on  personal 
knowledge  of  individuals  and  informal  networks  of 
communication;  and  4)  outdated  equipment. 

These  barriers  can  be  overcome  by  recognizing  the 
limitations  of  local  agencies:  identifying  cultural 
constraints  on  data  collection  and  interpretation; 
convincing  each  community  of  the  utility  of  a  public  health 
perspective  and  the  concomitant  modification  of  current 
data  collection  methods;  and  increasing  local  control  over 
the  information.  Indian  researchers  are  now  considered 
vital  to  the  development  of  epidemiological  research 
projects  in  Indian  communities.  Indian  researchers  can 
serve  as  cultural  liaisons  between  professional  and  lay 
communities.  This  is  particularly  important  when  research 
involves  marginal  populations,  issues  that  are  highly 
associated  with  fear  and  stigmatization,  and  where  study  in 
a  clinical  setting  is  difficult  or  impossible. 

Development  of  Research  Methods  for  Small  Trihal 
Populations 

As  the  system  of  health  care  for  Indian  people 
changes  undercurrent  federal  re-organization  the  need  for 
planning  at  the  community  level  is  increasingly  apparent. 
Programs  planned  at  the  national  and  regional  levels  do  not 
effectively  address  local  needs  and  priorities.  However, 
until  very  recently,  the  IHS  has  been  the  primary  source  of 
program  planning,  program  development,  and  program 
funding. 

In  order  to  justify  the  expenditure  of  federal  funds 
for  program  development,  the  IHS  requires  calculated 
morbidity  and  vital  event  rates.  These  rates  are  rarely 
calculated  below  a  regional  level  because,  for  small 
geographic  areas,  the  number  of  events  are  often  quite 
small.  (The  general  consensus  is  that  in  small  areas  the 
number  of  events  may  be  affected  by  yearly  fluctuations 
which  are  purely  random  and  a  given  rate  in  any  one  year 
may  be  very  different  from  the  true  rate.)  Although  a  small 
number  of  events  may  be  insufficient  for  determining  rates 
the  events  are  significant  to  the  local  community  and, 
proportionally,  may  have  an  enormous  impact  on  health  of 
the  community  and  the  direction  of  program  planning. 

Tribal  communities  need  information  that  is 
adequate  for  identifying  the  scope  of  local  health  issues. 


sufficient  for  community  feedback,  and  collected  in  a 
format  that  addresses  not  onlj  multiple  levels  ol  need  lull 
the  varying  needs  of  different  agencies  In  addition,  in 
small  communities  the  process  of  obtaining  and  sharing 
information  must  avoid  breaches  of  confidentiality  on  the 
individual  level  and  avoid  stigmatization  on  the  community 
level. 

The  need  for  information  goes  beyond  acquiring 
incidence  and  prevalence  rates.  If  tribes  are  to  be  able  to 
understand  the  scope  of  local  issues  and  develop 
appropriate  strategies  for  response  they  must  be  able  to 
document  not  only  the  existence  of  a  problem  but 
contributing  factors  and  related  problems.  This  is  best 
achieved  by  using  an  epidemiological  approach  to  research 
and  data  collection. 

Epidemiologic  objectives,  a  natural  result  of  a  public 
health  perspective,  should  set  the  requirements  for  study 
design  and  data  analysis.  Although  statistical  hypothesis 
testing  is  generally  felt  to  be  the  preferred  method  for 
research,  it  is  a  mode  of  analysis  that  offers  less  insight 
into  epidemiological  data  than  alternative  methods  that 
emphasize  estimation  of  interpretable  measures. 
Compromises  in  study  design  or  analysis  cannot  be 
defended  in  pursuit  of  a  statistical  goal  or  to  use  a 
statistical  method  that  does  not  accomplish  the  study 
objectives.  Communities  need  useable  information  that  is 
relevant  to  local  conditions.  According  to  Rothman,  1986, 
the  fundamental  task  is  to  quantify  the  occurrence  of  a 
public  health  issue,  evaluate  causal  and  sequelae 
hypotheses,  and  relate  the  occurrence  to  the  community's 
social,  cultural,  and  political  context. 

Analytical  Issues 

When  rates  are  the  issue  numerators,  denominators, 
confidence  intervals,  and  statistical  significance  are  key 
considerations.  The  confidence  interval  is  the  most 
common  method  used  to  assess  the  adequacy  of  an 
observed  rate,  as  an  estimation  of  its  true  value.  The 
general  rule  for  data  sets  is  that  rare  events  in  small 
populations  mean  wide  confidence  intervals.  The 
confidence  interval  is  the  most  common  method  used  to 
assess  the  adequacy  of  an  observed  rate  as  an  estimation  of 
its  true  value  (D*Angelo,  1993).  In  general,  a  95% 
confidence  interval  is  defined  as  a  95%  probability  that  the 
"true  rate"  is  included  in  the  interval.  Since  any  rate  based 
on  fewer  than  20  events  in  the  numerator  will  have  a  95% 
confidence  interval  about  as  wide  as  the  rate  itself,  it  is 
preferred  not  to  calculate  rates  involving  fewer  than  20 
events.  The  reality  is  that  20  events  or  less  is  common  in 
small  communities. 

There  are  generally  thought  to  be  two  options  for 
increasing  the  numbers  and  thus  narrowing  the  confidence 
interval:  1 )  combine  several  years  of  data;  or  2)  combine 
data  from  several  smaller  geographic  areas  into  larger 
ones.  The  norm  for  the  IHS  is  to  combine  three  years  of 
data  at  the  Area  (regional)  level.  This  is  sufficient  to  look 
for  trends  but  not  end  up  with  misleading  rates  due  to 
changing  local  conditions.  The  IHS  doesn't  report  rates 
with  small  numbers  because  they  are  basically 
meaningless— as  likely  to  be  statistical  noise  as  they  are  to 
be  valid  estimates.  However  an  individual  community 
finds  its  own  numbers  very  useful.  If  these  number  can  be 
presented  in  a  simple  but  meaningful  way,  communities 
can  not  only  begin  to  effectively  address  the  problems  but 
can  also  develop  methods  for  future  data  collection  and 
analysis. 

There  is  also  a  third  option—  100%  ascertainment, 
that  is  including  every  reported  incident.  With  100% 
ascertainment  confidence  intervals  are  not  important  and, 
whether  the  data  are  statistically  significant  or  not  is  not  the 
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point.  What  you  have  are  actual  numbers  and  a  "true  rate" 
that  the  community  can  understand  and  can  use  for 
planning  purposes.  This  is  much  more  important  to  tribes 
than  statistical  significance  or  statistical  comparisons. 

Although  seemingly  obvious  for  small  areas,  100% 
ascertainment  it  is  not  always  easily  accomplished.  With 
the  involvement  of  multiple  agencies,  some  degree  of 
coordination  and  standardization  of  data  collection  is 
critical  to  increase  precision  and  reduce  random  error. 
While  the  best  way  to  increase  precision  is  to  increase  the 
sample  size,  this  is  not  realistic  in  small  communities. 

The  alternative  is  to  improve  the  design  of  the  study 
to  increase  the  efficiency  with  which  information  is 
obtained.  With  child  abuse  and  neglect  data,  the  local 
multidisciplinary  child  protection  teams  can  facilitate  this 
process.  Because  key  response  agencies  are  represented 
on  the  team,  each  can  ensure  that  the  needs  of  their  agency 
are  being  met  and  that  the  data  collection  process  is  not  an 
added  burden  to  existing  data  collection.  When  planning  is 
the  focus,  as  opposed  to  calculating  rates,  the  utility  of 
100%  ascertainment,  and  the  need  for  coordinated  data 
collection,  becomes  apparent  to  tribal  agencies. 

The  first  step  is  to  coordinate  data  collection 
between  different  agencies  to  ensure  that  all  cases  are 
included  and  that  duplicates  are  removed.  It  may  be 
necessary  to  simplify  protocols  so  each  agency,  regardless 
of  the  sophistication  of  the  data  management  system,  can 
provide  useful  information.  A  data  collection  instrument 
specific  to  child  abuse  caseloads,  and  Epi-info,  a  statistical 
software  program  developed  by  the  CDC,  proved  to  be 
useful  tools  for  some  of  the  tribes. 

It  is  also  important  to  simplify  the  analysis  and 
return  a  maximum  amount  of  understandable  information 
to  each  community  as  well  as  to  the  IHS  area  level.  Bar 
graphs,  and  simplified  analytic  methods  are  easily 


understood  and  interpreted  by  tribal  service  providers  and 
program  planners.  The  ability  to  identify  local  trends  and 
compare  information  with  other  area  tribes  is  essential.  By 
using  a  simple  system  compatible  with  the  local  level  of 
sophistication  and  available  equipment  tribal  communities 
can  have  information  far  more  valuable  than  complex 
statistical  analyses. 

Results 

A  test  of  revised  methods  for  data  collection  and 
small  area  analysis  was  conducted  in  one  regional  area  and 
included  16  tribal  communities.  A  data  collection 
instrument  was  developed  and  information  on  all  cases  for 
a  one  year  period  were  collected  by  local  Child  Protection 
Teams.  The  16  communities  reported  a  total  of  379 
incidents  of  child  abuse  and  neglect.  The  number  of  cases 
reported  ranged  from  a  low  of  one  case  to  a  high  of  79 
cases.  (Information  from  six  of  the  communities, 
accounting  for  287  incidents,  are  included  in  this  report.) 

'"True"  rates  were  reported  back  to  each  tribe  along 
with  descriptive  information  and  a  simple  analysis.  The 
information  was  also  compared  with  an  area  aggregate  and 
the  national  estimates  from  the  IHS  project.  By  presenting 
individual  community  and  area  data  in  a  comparative 
graphic,  local  providers  could  assess  the  scope  of  local 
problems.  The  result  was  that  the  tribes  finally  had 
information  that  was  useful  for  understanding  local 
problems,  comparing  notes  with  other  tribes,  and  an 
opportunity  to  assess  the  use  of  limited  resources. 

Although  individual  communities  had  the  actual 
numbers  to  work  with,  comparing  percentages  between 
communities  helped  identify  issues  that  may  be  unique  to 
each  community  and  issues  that  every  tribe  needed  to 
address.  In  addition,  in  group  presentations,  percentages 
ensured  the  confidentiality  of  individual  cases. 


Figure    1.    Percent   of  reported   incidents   by   abuse   type- 
comparison    between    six    tribal    communities,    the    regional 
area,    and   the    national    estimate 


90  -f 


to 

70 -j 

c 

0) 

- 

■o 

60- 

u 

_ 

c 

50- 

42 


As  indicated  in  figure  I .  the  i>  pe  of  abuse  reported 
varied  widely  between  communities  The  black  horizontal 
lines  indicate  the  national  level  estimate;  the  first  bar 
indicates  the  aggregate  estimates  for  the  area;  and  the  other 
bars  indicate  the  percent  of  the  total  number  of  incidents 
reported  by  each  community  by  the  type  ol  abuse  reported. 
The  difference,  whether  statistically  significant  or  not, 
helped  individual  communities  assess  issues  such 
reporting  biases  which  might  account  for  numbers  that 
were  lower  than  expected,  and  risk  factors  that  might 
account  for  numbers  that  were  higher  than  expected.  For 
example,  the  national  estimate  of  sexual  abuse  is  28%  of 
all  reported  cases.  The  combined  average  for  all 
communities  in  the  area  was  10%.  One  community 
reported  no  cases  of  sexual  abuse  while  another 
community  reported  34%  of  all  their  cases  were  identified 
as  sexual  abuse.  Each  community  felt  the  need  to  address 
the  issue  differently  and  a  strategy  based  on  either  28%  or 
10%  would  not  likely  have  met  the  needs  of  either. 

Some  of  the  variables  showed  few  if  any 
differences.  This  was  also  very  useful  information  because 
it  confirmed  the  pervasiveness  of  certain  risk  factors.  For 
example,  figure  2  indicates  that,  for  Indian  children,  home 
is  the  most  likely  location  where  abuse  will  occur.  This 
was  very  important  information  for  the  tribes  to  have. 
Although  local  services  providers  have  long  known  that 
child  abuse  is  a  family  issue,  national  attention  has  been 
primarily  focused  on  the  abuse  of  Indian  children  in 
government  schools.  This  level  of  information  helped 
tribal  agencies  re-direct  attention  towards  the  needs  of 
families. 


Figure  2.  Percent  of  reported  incidents  occurring  at  the 
victim's  home— community,  area,  and  national  Dcomparison. 
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Figure  3  indicates  the  relationship  between  the 
victim  and  the  offender  for  reported  incidents.  In  general, 
parents  and  close  biological  relatives  were  identified  as  the 
primary  perpetrators.  Mothers  were  most  often  identified 
with  incidents  of  neglect,  fathers  with  incidents  of  physical 
abuse,  and  biological  relatives  with  incidents  of  sexual 
abuse.  Again  this  information  was  critical  to  focusing 
attention  on  the  family  and  the  extended  family.  Within 
each  community,  numbers  that  were  higher  than  expected 
or  lower  than  expected  led  to  discussion  of  reporting 
issues,  risk  factors,  and  program  planning. 


Figure    3.    Percent    of    reported    incidents    by    victim-offender 
relationship— comparison    of    community,    area,    and    national 
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A  major  issue  for  Indian  people  is  substance  abuse. 
There  was  an  assumption  that  all  incidents  of  child  abuse 
were  the  result  of  substance  abuse.  The  data  showed 
however,  that  while  the  correlation  between  perpetrator 
substance  use  and  child  abuse  were  higher  for  Indian 
people  than  for  the  general  population,  it  was  not  a  100% 
correlation.  If  fact,  the  proportion  of  incidents  involving 
substance  abuse  on  the  part  of  the  perpetrator  varied  with 
the  type  of  abuse,  age,  sex,  and  the  relationship  of  the 
perpetrator  to  the  victim. 

As  indicated  in  figure  4,  while  most  of  the 
communities  were  close  to  the  national  estimate  (72%)  and 
the  area  estimate  (67%),  the  actual  numbers  of  incidents 
involving  substance  abuse  varied  from  a  high  of  89.9%  to 
a  low  of  51%.  Although  substance  abuse  is  a  significant 
factor  in  incidents  of  child  abuse  in  all  the  communities, 
some  communities  may  need  to  more  aggressively  address 
the  relationship  between  substance  abuse  and  child  abuse. 
This  correlation  also  has  indications  for  treatment  and 
prevention. 

The  sum  total  of  the  data  has  had  important 
implications  for  community  planning  and  service  provision 
at  the  local  level.  The  information  obtained  through  this 
process  increased  the  tribes  awareness  of  the  issues, 
facilitated  local  level  planning,  justified  requests  for 
services  from  the  IHS,  and  in  combination,  helped  federal 
and  state  agencies  guide  planning  and  appropriations. 


In  order  to  obtain  needed  data,  100%  ascertainment 
over  a  period  of  time  can  prove  to  be  an  effective  means  of 
evaluating  the  scope  of  local  problems.  While  this  may 
require  coordination  between  agencies  and  redefining  data 
collection  protocols,  the  benefit  of  this  approach  for  tribal 
communities  includes  community  education  and 
empowerment,  better  future  planning,  and  a  stronger  focus 
on  individual  community  needs. 
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Conclusion 

The  future  of  Indian  health  care  is  uncertain.  The 
US  trust  responsibility  to  tribes  and  recent  Indian 
legislation  is  in  jeopardy.  The  Indian  Health  Service  is 
facing  regressive  budgets  and  massive  reorganization. 
Tribes  need  to  be  able  to  address  public  health  issues  in 
their  communities.  In  order  to  accomplish  this  task,  the 
current  focus  of  data  collection,  methods  of  data  analysis, 
and  the  of  data  need  to  be  reconsidered. 

In  Indian  communities  the  need  for  planning 
supersedes  the  need  for  statistical  comparisons.  When  data 
on  a  large  scale  is  available  it  can  provide  a  "standard"  by 
which  individual  communities  can  assess  local  problems 
but  it  can  not  be  a  substitute  for  local  level  information. 
Understanding  similarities  and  differences  between  local 
and  regional  or  national  number  help  small  communities 
prioritize  issues  and  focus  limited  resources. 
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1 .   INTRODUCTION 

The  Special  Supplemental  Nutrition 
Program  for  Women,  Infants,  and  Children 
(WIC)  is  a  federal  grant  program  for 
states  and  Indian  Tribal  Organizations 
(ITOs)  that  is  administered  by  the  Food 
and  Consumer  Service  (FCS)  of  the  U.S. 
Department  of  Agriculture  (USDA) .  The 
program  provides  nutrition  and  health 
assistance  services  for  low- income 
childbearing  women,  infants,  and 
children.  Pregnant  and  postpartum  women 
and  children  under  the  age  of  5  who  have 
family  incomes  at  or  below  185  percent  of 
the  applicable  poverty  guidelines 
($27,380  for  a  family  of  four  as  of  July 
1,  1994)  are  "income-eligible"  to 
participate.  To  be  fully  eligible, 
participants  must  be  determined  to  be  at 
nutritional  risk  based  on  a  medical 
and/or  nutritional  risk  factor  identified 
by  a  competent  health  professional. 
Persons  exhibiting  a  medical  risk  factor 
(such  as  anemia,  underweight,  or 
diabetes)  receive  higher  priority  than 
persons  at  risk  of  inadequate  nutrition. 

WIC  participants  receive  a  federally- 
prescribed  package  of  foods  designed  to 
meet  their  specific  nutritional  needs. 
For  example,  a  WIC  package  for  a  newborn 
infant  would  consist  entirely  of  infant 
formula,  while  a  child's  package  would 
contain  such  items  as  milk,  cheese, 
peanut  butter,  cereal,  and  juice.  WIC 
participants  also  receive  nutrition 
education  and  counselling  and  access  to 
health  and  social  services. 

Research  has  demonstrated  that  WIC 
has  been  successful  in  reducing  infant 
deaths,  low  birthweight,  premature 
births,  and  other  health  problems.  The 
program  has  also  improved  the  nutrition 
and  health  care  use  of  participants. 

WIC  is  not  an  entitlement  program- - 
the  number  of  participants  in  each  year 
is  limited  by  the  funds  appropriated. 
Funds  are  allocated  among  states  based  on 
two  formulas  established  by  regulation, 
one  for  food  and  one  for  administration 
and  nutrition  services.  The  food  funding 
formula  has  been  modified  several  times 
since  the  program' s  inception  in  the 
early  1970s  to  reflect  changes  in  the 
size  of  the  program  and  changing  program 
priorities . 

The  size  of  the  WIC-eligible 
population  in  each  state  has  always  been 
a  factor  in  determining  state  WIC  grants, 
and  the  availability  of  timely  and 
accurate  data  on  the  number  of  women, 
infants,  and  children  income-eligible  for 
WIC  has  long  been  an  issue  of  concern. 
However,  the  relative  importance  of  these 
data  and  other  components  in  the  funding 


formula  has  varied  over  time. 

President  Clinton's  first  budget 
highlighted  the  expansion  of  WIC  as  a 
major  priority,  and  set  forth  as  a  goal 
"fully  funding"  the  program,  that  is, 
providing  enough  funds  to  allow  all 
eligible  persons  who  want  to  participate 
in  the  program  to  do  so.  The  WIC  program 
has  already  expanded  significantly  in 
recent  years.  From  1989  to  1994,  annual 
WIC  appropriation  increases  ranged  from  9 
to  12  percent  per  year.  This  program 
growth,  coupled  with  the  expected  move 
toward  full  funding,  prompted  FCS  to 
revise  the  funding  formula  in  fiscal  year 
1994.  This  revision  included  a 
substantially  increased  emphasis  on  the 
eligibles  data  as  a  basis  for  funds 
allocation,  and  brought  the  need  for 
timely  and  accurate  data  to  the 
forefront . 

2.   EVOLUTION  OF  THE  WIC  FOOD 
FUNDING  FORMULA 

The  WIC  program  was  established  in 
1972,  and  for  the  first  several  years  of 
operation,  state  grants  were  determined 
at  the  discretion  of  the  USDA.  In  1979, 
a  food  funding  formula  was  formally 
established  by  regulation.  This  formula 
set  forth  two  essential  components  that 
were  the  primary  basis  for  food  funds 
distribution  until  1987.  First,  states 
were  provided  with  their  prior  year 
funding  level,  plus  some  adjustment  for 
inflation,  assuming  adequate  funds  were 
available.  Any  remaining  funds  were 
allocated  based  on  a  "growth" 
calculation,  which  attempted  to  direct 
funds  to  states  on  the  basis  of  need  for 
the  program.  Specifically,  the  formula 
considered  each  state's  share  of  the 
estimated  national  population  of  income- 
eligible  women,  infants,  and  children, 
and  its  relative  health  status,  as 
measured  by  the  state's  infant  mortality 
rate  and/or  low  birthweight  rate  relative 
to  the  national  average. 

The  original  19  79  funding  formula 
placed  equal  weight  on  the  income - 
eligibles  data  and  the  health  data. 
However,  in  19  84,  the  formula  was  revised 
to  place  much  greater  emphasis  on  the 
income -eligibles  data,  increasing  their 
weight  in  the  growth  calculation  to  a 
minimum  of  80  percent  (and  in  some  cases 
much  closer  to  100  percent)  .  This  change 
reflected  a  belief  that  the  eligibles 
data  best  indicated  relative  need  among 
states. 

In  1987,  a  major  change  occurred  when 
a  "targeting"  component  was  added  to  the 
food  funding  formula.  This  component 
reflected  concern  that  the  limited  WIC 
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funds  be  allocated  based  not  only  on  the 
estimated  need  for  the  program,  but  also 
on  states'  demonstrated  ability  to  serve 
those  at  highest  risk.  In  proposing  the 
new  targeting  component,  USDA  stated  its 
concern  that  the  existing  funding  formula 
did  not  discriminate  between  state 
agencies  that  had  targeted  this 
population  effectively  and  those  that  had 
not.  Thus,  the  1987  regulation  required 
that  after  stability  grants  were  made, 
half  of  any  additional  funds  be 
distributed  based  on  states'  shares  of 
the  national  population  of  "high 
priority"  participants  (defined  as  women, 
infants,  and  children  with  a  demonstrated 
medical  risk) .  The  remaining  half  was 
distributed  based  on  the  growth 
calculation  (that  is,  using  the  states' 
shares  of  the  eligible  population  and  the 
state  health  indicators) .  States  with 
well -targeted  programs  according  to  the 
new  measure  also  received  a  larger 
inflation  adjustment. 

In  1994,  USDA  determined  that  several 
components  of  the  1987  rule  had  become 
outdated.  In  close  consultation  with  the 
states,  USDA  concluded  that  the  program 
had  expanded  so  significantly  that  it  was 
no  longer  necessary  to  provide  incentives 
for  states  to  serve  those  at  highest 
risk.  In  addition,  the  targeting 
component  was  viewed  as  an  obstacle  to 
achieving  funding  equity  among  states. 
Thus,  the  1994  funding  rule  eliminated 
both  the  participation-based  targeting 
component,  as  well  as  the  portion  of  the 
growth  component  that  measured  relative 
health  status.  The  new  rule  stated  that 
after  stability  grants  had  been  provided, 
all  remaining  funds  were  to  be  allocated 
based  on  states'  estimated  shares  of  the 
national  eligible  population. 

The  new  funding  formula  was 
structured  to  allocate  any  growth  funds 
to  states  whose  current  resources  were 
less  adequate  for  serving  their  estimated 
eligible  populations  relative  to  other 
states.  This  is  accomplished  using  a 
"fair  share"  concept.  A  state's  fair 
share  of  available  funds  is  its  share  of 
the  estimated  national  population  of 
persons  eligible  for  the  program.  Thus, 
a  state  with  one  percent  of  the  eligible 
persons  has  a  fair  share  of  one  percent 
of  total  available  food  funds.  States 
whose  stability  grants  are  less  than 
their  fair  shares  receive  growth  funds. 
The  amount  of  growth  funds  received  by  an 
"under  fair-share"  state  is  directly 
proportional  to  the  difference  between 
the  stability  grant  and  the  fair  share. 
States  with  stability  grants  in  excess  of 
their  fair  shares  do  not  receive  growth 
funds  (unless  all  the  "under  fair- share" 
states  decline  to  accept  the  full  amount 
of  growth  funds  available) . 

3.   WIC  ELIGIBLES  DATA  FOR  STATES 

The  establishment  of  the  fair  share 
component   of   the   funding   formula, 


combined  with  the  rapid  growth  of  the 
program,  heightened  USDA' s  concern  about 
the  quality  of  state  eligibles  estimates. 
Under  previous  rules,  census  data  were 
specifically  identified  as  the  source  for 
calculating  states'  shares  of  the 
eligible  population.  Data  from  the  1980 
census  were  used  from  the  early  1980s 
until  1994,  when  1990  census  data  were 
used.  WIC- eligible  infants  and  children 
were  estimated  directly  from  census 
counts  of  infants  and  children  at  or 
below  185  percent  of  the  poverty  line. 
Income- eligible  pregnant  and  postpartum 
women  were  estimated  indirectly  based  on 
the  counts  of  income- eligible  infants. 

The  lack  of  timeliness  of  the  census 
data  was  a  longstanding  concern,  and  the 
1987  funding  rule  noted  that  the  census 
data  were  flawed  in  this  respect. 
However,  no  alternative  data  source  was 
put  forth.  In  the  1994  rule,  no 
specific  source  of  WIC  eligibles  data  was 
identified  in  order  "to  allow  for  the  use 
of  the  most  timely  and  reliable  data  as 
it  becomes  available."  In  discussions 
with  the  states,  USDA  committed  to  place 
high  priority  on  developing  an 
alternative  source  to  the  census  for  the 
eligibles  data.  State  WIC  agencies 
desired  improvements  in  the  eligibles 
data  not  only  for  allocating  funds,  but 
also  for  using  the  data  as  a  benchmark  to 
assess  program  performance. 

FCS,  through  contract  research, 
undertook  to  develop  a  new  methodology 
for  estimating  the  population  income - 
eligible  for  WIC  in  each  state.  The 
research  focused  on  the  Current 
Population  Survey  (CPS)  as  the  likely 
alternate  to  the  census  for  state-level 
income  data.  FCS  identified  several 
criteria  for  the  state-level  eligibles 
estimates,  including:  consistent 
methodology  and  data  sources  across 
states;  an  understandable,  technically 
sound  methodology;  an  ability  to  update 
the  estimates  annually  and  to  capture 
year-to-year  changes  in  states'  relative 
positions;  and  the  use  of  data  that  are 
as  current  as  possible. 

4.   EVOLUTION  OF  THE  ESTIMATION 
METHODOLOGY 

The  methodology  for  estimating  WIC 
eligibles  has  evolved  more  slowly  than 
the  formula  for  allocating  WIC  funds. 
The  first  generation  estimator,  which 
used  census  data,  was  essentially 
unchanged  from  1979  to  1994.  It  was  just 
last  year  that  a  second  generation 
estimator  was  implemented,  although  a 
third  generation  estimator  is  currently 
under  development.  The  next  three 
sections  of  this  paper  describe  the  first 
and  second  generation  WIC  eligibles 
estimators  and  briefly  introduce  the 
third  generation  estimator. 

4.1  The  First  Generation  Estimator 

According  to  the  first  generation  WIC 
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eligibles  estimator,  the  current  number 
of  eligibles  is  the  same  as  the  number 
measured  from  the  most  recent  census  data 
available.  Although  census  estimates  are 
derived  from  large  samples  and, 
therefore,  are  precise,  there  was 
widespread  dissatisfaction  with  the  first 
generation  estimator  among  policy 
analysts,  WIC  administrators,  and 
advocates.  They  believed  that  census 
estimates  were  very  inaccurate,  failing 
to  account  for  often  rapidly  changing 
economic  conditions.  According  to  census 
and  CPS  data,  the  recession  of  1990-1991 
was  accompanied  by  a  20  percent  increase 
nationwide  in  the  number  of  eligible 
infants  and  children  between  1989  (the 
year  to  which  census  income  data  pertain) 
and  1992.  That  increase,  amounting  to 
nearly  1.5  million  infants  and  children, 
cannot  be  explained  by  population  growth. 
The  percentage  of  infants  and  children 
eligible  for  WIC  rose  by  nearly  6 
percentage  points- -from  about  3  8  percent 
to  44  percent. 

Such  rapid  growth  in  eligibles 
certainly  creates  difficulties  for 
program  planning  and  performance 
monitoring,  key  uses  of  WIC  eligibles 
estimates.  However,  census  data  could 
still  provide  accurate  estimates  of  fair 
shares  for  distributing  program  funds  if 
the  numbers  of  eligibles  grew  by  the  same 
proportion  in  every  state.  But,  the 
recent  growth  in  eligibles  seems  to  have 
been  spread  very  unevenly  across  the 
states.  According  to  the  1990  census  and 
the  March  1993  CPS,  the  percentage  of 
infants  and  children  eligible  rose 
between  1989  and  1992  by  over  8 
percentage  points  in  Florida,  New  York, 
California,  and  New  Jersey,  but  by  under 
3  percentage  points  in  Massachusetts, 
Ohio,  Michigan,  and  Pennsylvania. 

These  figures  strongly  suggest  that 
by  assuming  nothing  is  changing  over 
time,  the  first  generation  estimator  may 
be  badly  biased.  As  an  alternative  to 
the  census,  the  CPS  provides  the  most 
timely  data  for  developing  annual  state 
estimates  of  WIC  eligibles.  However, 
despite  their  timeliness,  direct  CPS 
estimates  are  imprecise  because  state 
samples  of  infants  and  children  are  small 
in  all  but  a  few  states.  This 
fundamental  problem  of  small  area 
estimation- -the  lack  of  data,  that  is, 
the  small  number  of  sample  observations- - 
led  to  the  original  adoption  and 
continued  use  of  the  first  generation 
estimator,  an  "indirect"  estimator  that 
"borrows  strength"  from  the  past. 
However,  it  seems  that  the  gain  in 
precision  from  using  census  data  comes  at 
the  potential  cost  of  substantial  bias. 
Therefore,  we  sought  an  alternative 
estimator  to  minimize  the  tradeoff 
between  bias  from  lack  of  timeliness  and 
imprecision  from  lack  of  data.  Based 
partly  on  the  findings  of  Schirm  (1994) , 
who  assessed  the  relative  accuracy  of 


several  different  estimators  of  state 
poverty  rates,  we  began  development  of  a 
second  generation  estimator  that  uses 
"shrinkage"  methods. 

4.2  The  Second  Generation  Estimator 

The  second  generation  WIC  eligibles 
estimator  is  a  Bayesian  shrinkage 
estimator  that  optimally  averages  CPS 
direct  sample  estimates  and  predictions 
from  a  regression  model.  As  we  will  see, 
the  shrinkage  estimates  obtained  are  more 
timely  than  census  estimates,  and 
substantially  more  precise  than  CPS 
estimates.  This  section  describes  our 
eight -step  procedure  for  estimating  the 
numbers  of  infants  and  children  who  were 
income  eligible  for  WIC  in  each  state  in 
1992.  Additional  technical  details  can 
be  found  in  Schirm  (1995) . 

Step  1:  From  the  most  recent  census 
(1990) ,  derive  state  estimates  of  the 
percentage  of  infants  and  children  who 
were  income  eligible.  Because  the  family 
income  data  collected  in  the  census 
pertain  to  the  preceding  calendar  year, 
the  eligibility  estimates  are  for  1989. 
We  estimated  the  percentages,  rather  than 
the  numbers,  of  infants  and  children  who 
were  income  eligible  to  standardize  for 
state  population  size. 

Because  census  samples  for  states  are 
very  large,  the  estimates  are  precise. 
However,  they  quickly  become  "old"  if 
economic  conditions  have  changed 
substantially  since  the  census. 

Step  2 :  From  the  most  recent  CPS 
(March  1993) ,  derive  state  sample 
estimates  of  the  percentage  of  infants 
and  children  who  were  income  eligible. 
The  most  recent  CPS  that  has  income  data 
for  families  provides  more  timely 
information  than  the  census.  That  CPS 
was  the  March  1993  CPS  when  we  were 
developing  eligibles  estimates  to  be  used 
in  allocating  funds  for  fiscal  year  1995. 
Like  the  census,  the  CPS  collects  family 
income  data  for  the  prior  year.  Thus, 
the  sample  estimates  pertain  to  1992. 

Although  timely  compared  with  the 
census  estimates,  the  CPS  sample 
estimates  are  relatively  imprecise.  The 
standard  errors  for  the  CPS  estimates 
tend  to  be  large,  so  our  uncertainty  is 
great.  For  example,  according  to  widely 
used  statistical  standards,  we  can  be 
confident  only  that  the  percentage  of 
income-eligible  infants  and  children  in 
Delaware  was  between  22.5  percent  and 
41.6  percent.  This  range  is  so  wide  and 
our  uncertainty  so  great  because  the  CPS 
samples  of  infants  and  children  in  each 
state  are  small.  Indeed,  that  is  why  we 
derived  an  eligibility  estimate  for 
infants  and  children  combined,  rather 
than  separate  estimates,  one  for  infants 
and  one  for  children.  In  the  March  1993 
CPS,  there  are  data  for  fewer  than  30 
infants  for  most  states. 

Step  3:  Construct  sample  estimates  of 
the  change  in  the  percentage  eligible 
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between  1989  and  1992.  A  sample  estimate 
of  the  change  in  the  percentage  eligible 
between  19  89  and  1992  was  calculated  by- 
subtracting  the  census  estimate  for  1989 
from  the  CPS  estimate  for  1992.  We 
calculated  sample  estimates  of  change  for 
use  in  the  regression  and  shrinkage 
estimation  described  in  the  next  few 
steps.  Focusing  on  the  change  in  the 
percentage  eligible  between  1989  and 
1992,  rather  than  just  the  percentage 
eligible  in  1992,  is  a  simple  way  to 
reflect  a  strong  systematic  relationship: 
states  with  a  high  percentage  eligible  in 
1989  tend  to  have  a  high  percentage 
eligible  in  1992,  and  states  with  a  low 
percentage  eligible  in  1989  tend  to  have 
a  low  percentage  eligible  in  1992.  In 
principle,  our  shrinkage  method  obtains 
better  estimates  by  using  information  on 
not  only  where  a  state  "is,"  but  also 
where  it  "began." 

Step  4:  Using  a  regression  model, 
predict  the  change  in  the  percentage 
eligible  for  each  state  based  on  observed 
changes  in  (i)  Food  Stamp  Program  (FSP) 
participation,  (ii)  Unemployment 
Insurance  (UI)  Program  participation,  and 
(iii)  per  capita  income.  The  main 
limitation  of  the  sample  estimates 
derived  in  the  previous  step  is 
imprecision.  Regression  can  reduce  that 
imprecision.  Regression  estimates  are 
predictions  based  on  nonsample  or  highly 
precise  sample  data,  such  as  census  and 
administrative  records  data.  The  latter 
include  government  program  case  files  and 
vital  statistics  records. 

Regression  estimates  are  points  on  a 
regression  line,  a  line  obtained  by 
regressing  the  sample  estimates  from  the 
previous  step  on  predictor  variables. 
The  three  predictor  variables  we  used 
measure  the  changes  between  19  89  and  1992 
in  (1)  FSP  participation,  (2)  UI  Program 
participation,  and  (3)  per  capita  income. 
These  three  were  selected  as  the  best 
predictors  from  a  longer  list.  As 
expected,  the  estimated  regression  shows 
that  states  with  relatively  large 
increases  in  FSP  and  UI  Program 
participation  and  large  decreases  in  per 
capita  income  tend  to  have  relatively 
large  increases  in  the  percentage  of 
infants  and  children  eligible  for  WIC. 
The  standard  errors  for  regression 
estimates  are  much  smaller  than  the 
standard  errors  for  sample  estimates. 

Comparing  how  the  sample  and 
regression  estimators  use  data  reveals 
how  the  regression  estimator  "borrows 
strength"  to  improve  precision.  When  we 
derived  sample  estimates  in  Step  3,  we 
used  only  data  from  Delaware  to  estimate 
the  change  in  the  percentage  of  infants 
and  children  eligible  for  WIC  in 
Delaware,  even  though  Delaware,  like 
nearly  all  states,  has  a_  small  CPS 
sample.  Deriving  regression  estimates  in 
this  step,  we  estimated  a  regression  line 
from  sample  and  administrative  records 


data  for  all  the  states  and  used  the 
estimated  line  (with  administrative 
records  data  for  Delaware)  to  predict  the 
change  in  WIC  eligibles  for  Delaware.  In 
other  words,  the  regression  estimator  not 
only  uses  the  sample  estimates  from  every 
state  to  develop  a  regression  estimate 
for  a  single  state  but  also  incorporates 
data  from  outside  the  sample,  namely, 
data  in  administrative  records  systems. 
The  regression  estimator  improves 
precision  by  using  more  data  to  identify 
states  with  sample  estimates  that  seem 
too  high  or  too  low  because  of  sampling 
error,  that  is,  error  from  drawing  a 
sample  that  has  a  higher  or  lower 
percentage  of  eligible  infants  and 
children  than  the  entire  state  population 
has.  For  example,  suppose  a  state  had 
experienced  stable  FSP  and  UI  Program 
participation  and  rising  per  capita 
income.  Our  regression  estimator  would 
predict  a  stable  or  declining  percentage 
of  eligible  infants  and  children, 
implying  that  a  sample  estimate  showing  a 
large  increase  in  WIC  eligibles  is  too 
high.  The  regression  estimate  will  be 
lower  than  the  sample  estimate  for  such  a 
state.  On  the  other  hand,  if  the  sample 
data  for  a  state  show  a  much  smaller 
increase  in  eligible  infants  and  children 
than  expected  in  light  of  the  observed 
changes  in  FSP  and  UI  Program 
participation  and  per  capita  income,  the 
regression  estimate  for  that  state  will 
be  higher  than  the  sample  estimate. 

Step  5:  Using  "shrinkage"  methods, 
average  the  sample  estimates  of  change 
and  the  predictions  of  change.  As  noted, 
the  limitation  of  the  sample  estimator  is 
imprecision.  The  limitation  of  the 
regression  estimator  is  bias.  Some 
states  really  have  larger  or  smaller 
increases  in  WIC  eligibles  than  we  expect 
(and  predict  with  the  regression 
estimator)  based  on  changes  in  FSP  and  UI 
Program  participation  and  per  capita 
income.  Such  errors  in  regression 
estimates  reflect  bias. 

These  limitations  arise  for  the 
following  reasons.  The  sample  estimator 
uses  only  sample  data  for  one  state  to 
obtain  an  estimate  for  that  state.  It 
does  not  use  sample  data  for  other  states 
or  administrative  records  data.  Although 
the  regression  estimator  borrows 
strength,  using  data  from  all  the  states 
and  administrative  records  data,  it  makes 
no  further  use  of  the  sample  data  after 
estimating  the  regression  line.  It 
assumes  that  the  entire  difference 
between  the  sample  and  regression 
estimates  is  sampling  error,  that  is, 
error  in  the  sample  estimate.  No 
allowance  is  made  for  prediction  error, 
that  is,  error  in  the  regression 
estimate.  Although  not  all,  if  any,  true 
state  values  lie  on  the  regression  line, 
the  regression  estimator  assumes  they  do. 

Using  all  of  the  information  at  hand, 
a   shrinkage   estimator   addresses   the 
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limitations  of  the  sample  and  regression 
estimators  by  combining  the  sample  and 
regression  estimates,  striking  a 
compromise.  A  shrinkage  estimator  takes 
a  weighted  average  of  the  sample  and 
regression  estimates.  We  calculated 
weights  using  Bayesian  methods,  as 
described  in  Schirm  (1995)  .  Generally, 
the  more  precise  the  sample  estimate  for 
a  state,  the  closer  the  shrinkage 
estimate  will  be  to  it.  The  larger 
samples  drawn  in  large  states  support 
more  precise  sample  estimates,  so 
shrinkage  estimates  tend  to  be  closer  to 
the  sample  estimates  for  large  states. 
Given  the  precision  of  the  sample 
estimate  for  a  state,  the  weight  given  to 
the  regression  estimate  depends  on  how 
well  the  regression  line  fits,  that  is, 
whether  we  could  find  good  predictors 
reflecting  why  some  states  have  larger 
increases  in  WIC  eligibles  than  other 
states.  The  shrinkage  estimate  will  be 
farther  from  the  regression  estimate  and 
closer  to  the  sample  estimate  when  we 
could  not  find  good  predictors  and  the 
regression  line  fits  poorly.  In 
contrast,  the  shrinkage  estimate  will  be 
closer  to  the  regression  estimate  and 
farther  from  the  sample  estimate  when  the 
regression  line  fits  well.  Striking  a 
compromise  between  the  sample  and 
regression  estimators,  the  shrinkage 
estimator  strikes  a  compromise  between 
imprecision  and  bias.  The  sample  and 
regression  estimates  are  optimally 
weighted  to  improve  accuracy  by 
minimizing  a  measure  of  error  that 
reflects  both  imprecision  and  bias.  By 
accepting  a  little  bias,  the  shrinkage 
estimator  may  be  substantially  more 
precise  than  the  sample  estimator.  By 
sacrificing  a  little  precision,  the 
shrinkage  estimator  may  be  substantially 
less  biased  than  the  regression 
estimator. 

Step  6:  Add  the  shrinkage  estimate  of 
the  change  between  1989  and  1992  to  the 
census  estimate  of  the  percentage 
eligible  in  1989  to  get  a  shrinkage 
estimate  of  the  percentage  eligible  in 
1992. 

Step  7:  Multiply  the  shrinkage 
estimate  of  the  percentage  eligible  by 
the  state  population  of  infants  and  the 
state  population  of  children  to  get 
preliminary  shrinkage  estimates  of  the 
numbers  of  eligible  infants  and  children. 
To  obtain  separate  estimates  for  infants 
and  children,  we  have  assumed  that  the 
percentage  of  infants  who  were  income 
eligible  in  a  state  is  the  same  as  the 
percentage  of  children  who  were  income 
eligible,  an  assumption  well -supported  by 
census  data.  Our  estimate  of  that 
percentage  was  obtained  in  Step  6 . 

To  obtain  estimated  numbers  from 
estimated  percentages,  we  require  state 
population  estimates  for  both  infants  and 
children.  The  population  estimates  we 
used  pertain  to  the  resident  population 


on  July  1,  1992  and  were  developed  by  the 
U.S.  Bureau  of  the  Census  from  census  and 
administrative  records  (mainly  vital 
statistics)  data.  These  estimates  are 
often  called  "independent"  estimates 
because  they  are  not  based  on  CPS  or 
other  sample  data. 

Step  8:  Control  the  preliminary  state 
shrinkage  estimates  of  the  numbers  of 
eligible  infants  and  children  to  sum  to 
the  national  totals  for  eligible  infants 
and  children  obtained  from  the  CPS.  The 
most  recent  national  CPS  sample  estimates 
are  typically  used  to  develop  the  budget 
for  the  WIC  Program.  To  obtain  final 
shrinkage  estimates  for  states  that  sum 
(aside  from  rounding  error)  to  the 
national  totals  from  the  most  recent  CPS 
(March  1993),  we  ratio  adjust  the 
preliminary  state  shrinkage  estimates. 
This  ensures  that  the  estimates  used  to 
allocate  funds  are  consistent  with  the 
estimates  generally  used  to  determine 
total  program  funding.  The  adjustments 
were  small,  with  ratios  of  about  0.99  and 
1.03  for  infants  and  children, 
respectively. 

Second  Generation  Estimates.  The 
strengths  of  the  shrinkage  estimates 
obtained  from  our  second  generation 
estimator  are  that  they  are  more  timely 
than  census  estimates  and  substantially 
more  precise  than  direct  CPS  estimates. 
As  documented  in  Schirm  (199  5) ,  the 
shrinkage  estimates  have  much  smaller 
standard  errors  and  much  narrower 
confidence  intervals  than  CPS  estimates. 
A  shrinkage  confidence  interval  is,  on 
average,  61  percent  narrower  than  the 
corresponding  direct  sample  confidence 
interval .  According  to  rough 
calculations,  that  is  about  the  same  gain 
in  precision  that  would  be  obtained  from 
increasing  the  sample  size  of  the  CPS 
from  under  60,000  households  to  nearly 
400,000  households- -a  6.5-fold  increase. 

While  using  a  shrinkage  estimator 
greatly  narrows  confidence  intervals  and 
reduces  our  uncertainty,  using  shrinkage 
estimates  makes  an  important  difference 
in  how  WIC  funds  are  distributed.  Table 
1  shows  that  there  are  several  large 
differences  in  fair  shares  when  the  fair 
shares  are  calculated  using  shrinkage 
rather  than  census  estimates.  Even  a 
small  difference  in  fair  shares  can 
affect  a  state's  WIC  grant,  however, 
because  the  funding  formula  contains  a 
threshold.  Specifically,  a  state 
receives  growth  funds  only  if  its 
stability  funding  is  below  its  fair 
share.  A  small  increase  in  a  state's 
fair  share  may  make  it  eligible  for 
growth  funds,  while  a  small  decrease 
might  make  it  ineligible. 

FCS  used  the  shrinkage  estimates  of 
infants  and  children  income-eligible  for 
WIC, in  1992  to  determine  state  WIC  food 
grants  for  fiscal  year  1995.  Over  $125 
million  in  growth  funds  have  been 
distributed. 


49 


4.3  The  Third  Generation  Estimator 

Our  second  generation  estimator 
borrows  strength  to  improve  precision. 
However,  there  is  more  strength  to  be 
borrowed.  The  second  generation 
estimator  uses  census  estimates  for  the 
"base"  year  (1989)  and  CPS  estimates  for 
the  "current"  year  (1992  for  the  most 
recent  set  of  estimates) .  Estimates  for 
intervening  years  are  not  used,  although 
CPS  data  for  obtaining  such  estimates  are 
available.  With  each  intervening  year, 
we  are  ignoring  more  information  that 
could  be  relevant.  An  unusually  large 
increase  in  WIC  eligibles  over  three 
years,  for  example,  would  be  more 
plausible  and  look  less  like  sampling 
error  if  it  appeared  to  consist  of  a 
series  of  modest  increases  rather  than 
two  small  decreases  followed  by  one 
enormous  jump.  Using  data  for  only  the 
base  and  current  years,  however,  the 
second  generation  estimator  cannot 
distinguish  those  two  patterns  of  change. 
Also,  with  the  second  generation 
estimator,  we  can  do  little  if  our 
regression  model  seems  to  persistently 
under-  or  overpredict  for  a  state,  short 
of  finding  a  predictor  variable  that 
explains  why  that  state  is  different  from 
all  the  other  states. 

An  advantage  of  Bayesian  shrinkage 
methods  is  that  they  allow  additional 
data  to  be  used  in  a  systematic,  rather 
than  ad  hoc,  way.  We  are  currently 
developing  a  third  generation  estimator 
that  is  both  domain  and  time  indirect, 
borrowing  strength  across  not  only 
states,  but  also  time.  We  anticipate 
that  the  state  WIC  eligibles  estimates 
for  1993  will  be  derived  from  census  data 
for  1989,  as  well  as  CPS  data  for  1990, 
1991,  1992,  and  1993.  Administrative 
records  data  for  all  five  years  (1989- 
1993)  will  also  be  used.  The  third 
generation  estimator  will  take  account  of 
correlations  among  sample  estimates  from 
different  years  and  correlations  among 
regression  model  prediction  errors  from 
different  years. 


5.0   CONCLUSION 

Since  its  inception,  the  WIC  program 
has  undergone  important  changes .  Over 
the  last  few  years,  funding  for  the 
program  and  the  number  of  program 
participants  have  risen  dramatically. 
The  formula  for  allocating  funds  to  the 
states  was  also  revised  significantly  to 
place  greater  emphasis  on  the  number  of 
eligibles  in  each  state  as  the  basis  for 
calculating  state  WIC  grants.  This  new 
emphasis,  coupled  with  rapid  program 
growth,  heightened  the  need  for  timely, 
accurate  state  estimates  of  WIC 
eligibles.  Responding  to  that  need,  we 
have  developed  a  Bayesian  shrinkage 
estimator  that  optimally  averages  CPS 
sample  estimates  with  predictions  of  WIC 
eligibles  from  a  regression  model.  The 
predictions  are  based  on  observed  changes 
in  government  program  participation  and 
other  indicators  of  socioeconomic 
conditions.  The  shrinkage  estimates 
obtained  are  more  timely  than  census 
estimates,  which  had  been  used  for  fund 
allocation  prior  to  the  use  of  shrinkage 
estimates  for  fiscal  year  1995,  and 
substantially  more  precise  than  direct 
CPS  estimates.  The  shrinkage  estimator 
improves  precision  by  borrowing  strength, 
using  data  from  all  the  states  to  derive 
each  state's  estimate.  We  are  currently 
developing  a  new  shrinkage  estimator  to 
take  account  of  even  more  information  and 
borrow  strength  across  both  space  and 
time.  Hopefully,  the  new  estimator  will 
provide  even  better  estimates  for 
allocating  WIC  funds  to  the  states. 
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Table  1.   Effect  of  Using  Shrinkage  Rather  than  Census  Estimates 

Change  in  Fair  Share 


(Percentage  Points! 
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Mental  Health  Issues  in 
Ambulatory  Care  Settings 


PATIENT  CARE  GUIDELINES:  ANXIETY  AND  DEPRESSION  SCREENING  ISSUES  IN  MEASUREMENT 

Lori  A.  Cornelius-Quast,  Mayo  Clinic,  Mayo  Foundation 
Jim  Naessens,  Charles  Darby,  Jennifer  Ferguson,  Greg  Angstman,  Steve  Hagedorn,  Priscilla  Van  Grevenhof 


Introduction 

The  1990 's  were  declared  the  "Decade  of 
the  Brain"  by  President  Bush  and  Congress.   We 
are  at  the  midpoint  of  a  new  era  in  brain  and 
behavioral  sciences.   Computer  technology 
accompanied  by  an  increase  in  behavioral 
research  has  produced  more  information  about 
the  root  causes  and  appropriate  treatment  of 
mental  illness.   Within  the  realm  of  mental 
illness  are  the  devastating  disorders  of 
anxiety  and  depression. 

Anxiety  disorder  and  depression  are 
misunderstood  and  underestimated  as  health 
problems.   7-12%  of  men  and  20-25%  of  women 
will  have  major  depression  in  their  lifetime 
according  to  the  Agency  for  Health  Care  Policy 
and  Research  (AHCPR)  Clinical  Practice 
Guideline  on  Depression.   The  National 
Institute  of  Mental  Health  states  that 
generalized  anxiety  disorders  affect  12.6%  of 
the  people  in  the  United  States. 

In  1993  the  U.S.  Department  of  Health  and 
Human  Resources  Public  Health  Services 
Depression  Guideline  Panel  estimated  that  the 
prevalence  of  major  depression  in  primary  care 
outpatients  is  4.8-8.6%,  while  the  prevalence 
rate  for  anxiety  disorder  is  6.5%  in  primary 
care.   However,  historically  primary  care 
providers  have  detected  only  one-third  to  one- 
half  of  patients  with  major  depressive 
disorder . 

Numerous  reports  in  the  literature 
suggest  that  primary  care  providers  under- 
diagnose and/or  under-treat  depressive 
conditions.  It  was  with  this  information  in 
mind  that  priority  was  placed  on  the 
development  of  clinical  practice  guidelines 
for  recognition  and  treatment  of  depression 
and  anxiety  disorders  in  the  primary  care 
setting . 

Clinical  guidelines  are  a  consenual 
product  developed  to  reflect  the  current  state 
of  knowledge  on  effective  and  appropriate  care 
based  on  research  and  clinical  experience. 
Each  guideline  is  periodically  reviewed  and 
updated  based  on  the  continual  changes  in 
scientific  information  and  available 
technology.   By  promoting  early  detection  and 
treatment  of  these  devastating  disorders, 
clinical  guidelines  are  intended  to  make 
important  contributions  to  the  health  of  our 
community,  lessen  the  costs  of  health  care, 
and  contribute  to  the  scientific  understanding 
of  these  common  and  very  disabling  conditions. 

Currently  two  guidelines  addressing 
anxiety  disorder  or  depression  are  being 
utilized  by  primary  care  providers.   A 
guideline  developed  by  the   Agency  for  Health 
Care  Policy  and  Research  (AHCPR)  deals 
specifically  with  detection,  diagnosis,  and 
treatment  of  depression  in  primary  care.   The 


Institute  for  Clinical  Systems  Integration 
(ICSI),  of  which  Mayo  is  a  participating 
member,  has  also  developed  guidelines  relating 
to  the  screening  and  treatment  for  anxiety 
disorder  and  depression  in  primary  care.   The 
Section  of  Health  Services  Evaluation  at  the 
Mayo  Clinic  Rochester  collaborates  with  ICSI 
in  the  development,  implementation,  and 
measurement  of  practice  guidelines. 

Once  a  guideline  is  fully  implemented, 
baseline,  in-process,  and  end-results  data  are 
collected  on  a  pre-determined  time  schedule. 
Baseline  data  is  collected  retrospectively.  It 
allows  the  medical  practice  to  be  measured 
prior  to  introduction  of  the  guideline.  In- 
process  data  allows  for  measurement  of  a 
practice  as  a  guideline  is  introduced, 
reviewed,  and  implemented  by  the  clinic  sites. 
End-results  data  identifies  any  significant 
change  in  practice  after  implementation  is 
completed.   All  data  is  analyzed  and  presented 
to  the  individual  implementation  sites  where 
it  can  be  used  to  explore  and  improve  the 
understanding  of  practice  patterns. 

This  paper  presents  baseline  data  for 
anxiety  disorder  and  depression  screening  and 
diagnosis  from  the  Mayo  Clinic  Rochester. 
Patient  populations  were  generated  from  three 
primary  care  sites  using  ambulatory  billing 
data.   We  focused  on  the  four  patient  groups 
which  according  to  the  Institute  for  Clinical 
Systems  Integration  guideline  are  at  high  risk 
for  depression  or  anxiety  disorder  (Institute 
for  Clinical  Systems  Integration,  1994). 

METHODS 

Patients  at  high  risk  for  anxiety 
disorder  or  depression  include  those  diagnosed 
with  irritable  bowel  syndrome,  fatigue,  sleep 
disorders,  and  patients  with  six  or  more 
visits  to  a  primary  care  site  within  six 
months.   Using  electronic  billing  data  from 
the  period  October  1994  through  December  1994, 
a  simple  random  sample  of  patients  were 
selected  by  appropriate  ICD-9  diagnosis  codes. 
The  selected  visit  must  have  been  a  new 
diagnosis  or  the  first  occurrence  of  the 
symptom  at  the  identified  site.   These  medical 
records  were  collected  and  reviewed  by  an  RN 
to  gather  the  data. 

Symptoms  present  differently  for  anxiety 
and  depression.  Therefore  the  screening 
criteria  used  was  specific  to  each  diagnosis. 
Positive  screening  criteria  for  depression  was 
documentation  on  the  visit  which  identified 
any  of  the  following  statements:  the  patient 
is  sad,  down,  blue  or  teary;  the  patient  has 
diminished  interest  in  usually  enjoyable 
activities;  or  the  patient  is  unable  to  have 
fun.   Many  variations  of  these  statements  were 
found  as  was  documentation  that  the  patient 
did  not  have  these  symptoms . 
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The  positive  screening  criteria  for 
anxiety  disorder  was  more  inclusive. 
Documentation  of  one  of  the  following  must 
have  occurred  on  the  visit  identified: 
excessive  worrying;  sudden  out-of-the-blue 
attacks  of  fear,  terror,  or  apprehension; 
attacks  of  anxiety,  panic,  or  severe 
embarrassment;  a  sense  of  impending  doom;  fear 
of  loosing  control;  a  fear  of  dying,  fainting, 
or  of  going  crazy;   identified  situations  or 
places  that  are  avoided;  anxiety  that  causes 
significant  distress  or  avoidant  behavior  in 
your  daily  life.   This  type  of  documentation 
was  seldom  present  in  the  patient  record. 

In  order  to  find  the  first  diagnosis  of 
the  identified  symptoms,  we  excluded  any 
patient  with  a  visit  for  the  symptom  of 
interest  within  the  previous  six  months.   We 
also  excluded  any  patient  with  an  anxiety 
disorder  or  depression  diagnosis  or  a  visit  to 
a  psychiatric  provider  in  the  previous  six 
months  for  all  four  groups.   In  keeping  with 
the  intent  of  the  ICSI  guideline,  all  patients 
aged  18-64  identified  as  having  irritable 
bowel  syndrome,  fatigue,  or  sleep  disorder 
were  sampled.  Up  to  3  0  patients  per  month  were 
selected  from  the  three  primary  care  sites  for 
the  multiple  visit  criteria. 

RESULTS 

Overall,  the  data  was  collected  on  274 
patients.  Of  this  population  22%  were  screened 
for  anxiety  disorder  or  depression,  and  27%  of 
those  screened  were  actually  diagnosed  as 
having  an  anxiety  disorder  or  depression.   Of 
this  sample,  70.8%  were  women.   The  average 
age  of  this  population  was  43.   The 
demographics  and  screening  results  from  each 
patient  group  are  presented  in  Table  1. 

Irritable  bowel  syndrome  was  defined  by 
ICD-9  code  564.1  (colon,  irritable).   We 
identified  51  patients  with  irritable  bowel 
syndrome  among  the  three  primary  care  sites 
during  October-December  1994  who  fulfilled  the 
criteria  for  this  guideline.  The  age 
distribution  ranged  from  19  to  63.   Only  8%  of 
these  patients  were  screened  for  anxiety 
disorder  or  depression;  and  from  those 
screened,  25%  were  diagnosed  with  anxiety 
disorder  or  depression. 


Fatigue  was  defined  using  ICD-9  code 
780.7  (malaise,  fatigue).   13-29%  of  patients 
with  a  complaint  of  chronic  fatigue  may  have 
anxiety  disorder  (Manu,  1991).   Also  fatigue 
is  the  seventh  most  common  symptom  in  primary 
care,  and  up  to  24%  of  all  patients  surveyed 
in  primary  care  clinics  indicate  that  fatigue 
is  a  problem  (Kroenke  et  al,  1988) .   We 
identified  125  patients  with  fatigue  who 
fulfilled  the  criteria  for  this  guideline.  The 
age  distribution  ranged  from  18  to  63.   Nearly 
34%  of  this  sample  were  screened  for  anxiety 
disorder  or  depression;  and  of  those  screened, 
24%  were  diagnosed  with  anxiety  disorder  or 
depression . 

The  symptom  of  sleep  disorders  had 
several  ICD-9  diagnosis  codes  of  which  we  used 
four:  780.50,  780.52,  780.55,  and  780.59. 
These  respectively  relate  to  unspecified  sleep 
disturbance;  insomnia  not  otherwise  specified; 
disruptions  of  24-hour  sleep-wake  cycle;  and 
other.   The  ICD-9  diagnosis  codes  dealing  with 
sleep  apnea  were  not  used.   Twenty  patients, 
who  fulfilled  the  criteria  for  this  guideline, 
were  identified  with  sleep  disorder.   Of  those 
identified,  5  5%  were  men.  This  was  the  only 
sample  that  contained  more  men  than  women. 
The  age  distribution  was  19-50.   One  half  of 
the  twenty  patients  identified  were  screened 
for  anxiety  disorder  or  depression,  and  of 
that  sample  two  were  diagnosed  with  anxiety 
disorder  or  depression. 

According  to  a  study  done  by  Weissman  and 
Klerman,  patients  with  undiagnosed  depression 
average  more  than  six  visits  per  year  with 
their  primary  care  provider (Weissman  and 
Klerman,  1977)  .   Patients  with  anxiety 
disorder  have  the  highest  risk  of  having 
multiple  medically  unexplained  symptoms  and  of 
being  high  utilizers  of  medical  ambulatory 
services  compared  to  people  with  and  without 
psychiatric  disorders  in  the  community  (Katon, 
1992).   This  is  a  valuable  population  to 
measure  for  anxiety  and  depression  screening. 

To  obtain  the  multiple  visit  sample  we 
examined  a  moving  window  of  six  months  in 
which  six  visits  at  any  primary  care  site  must 
have  occurred.  A  patient's  sixth  visit  defines 
the  site  at  which  the  guideline  is  applied  and 
measured.   Excluded  from  this  group  were  any 
patients  with  obstetric  or  nurse-only  visits. 


Table  1 


Irritable  bowel 
syndrome 

Fatigue 

Sleep  disorder 

Multiple  visits 

Sample  size 

n  =  51 

n  =  125 

n  =  20 

n  =  78 

Age:  Mean 

46.7 

41.9 

39.4 

41.9 

sd 

13.2 

12.1 

12.1 

12.3 

#  female 

82.4% 

70.4% 

45.0% 

70.5% 

#  screened 

7.8% 

33.6% 

50.0% 

5.1% 

#  diagnosed 

25.0% 

23.8% 

20.0% 

75.0% 
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To  eliminate  nurse-only  visits  such  as  blood 
pressure  checks,  injections,  or  throat 
cultures,  we  included  only  those  CPT-4  codes 
which  indicate  patient-physician  interactions 
and  excluded  minor  contacts.  78  patients  were 
identified  as  having  multiple  visits  who 
fulfilled  the  criteria  for  this  guideline.  The 
age  distribution  was  20  to  63.  Of  this  sample, 
only  5%  were  screened  for  anxiety  disorder  or 
depression;  and  of  that  population,  75%  were 
diagnosed . 

DISCUSSION 

Several  similarities  emerged  from  this 
data.   The  average  age  for  our  samples  ranged 
from  39  to  47.   Also  three  of  the  four  groups 
had  substantially  more  women  than  men 
(p=.021).   The  screening  rate  for  anxiety 
disorder  and  depression  was  very  low  except 
for  the  symptoms  of  sleep  disorder  and  fatigue 
(p<.001) .   Among  those  screened,  the  diagnosis 
rates  were  also  relatively  low. 

Some  limitations  also  became  apparent. 
The  sample  selection  was  based  entirely  on 
billing  diagnosis.  The  social  stigma  and 
ramifications  of  being  diagnosed  with  anxiety 
disorder  or  depression  may  lead  providers  to 
under  report  these  conditions.  The  method  on 
which  we  relied  to  determine  if  these  patients 
had  a  "new"  diagnosis  of  the  presenting 
symptom  was  no  billing  diagnosis  in  the  prior 
six  months.  Therefore  we  may  not  have  a 
representative  sample  of  the  primary  care 
population.  Also  we  did  not  have  a  gold 
standard  assessment  of  anxiety  or  depression 
and  cannot  assess  the  sensitivity  or 
specificity  of  our  measures.   We  had  no  prior 
data  to  use  as  a  comparison. 

Identifying  true  multiple  physician 
visits  was  difficult  within  our  system.   This 
population  was  easy  to  define  but  difficult  to 
operationalize.   Our  original  multiple  visit 
sample  included  patients   seen  only  by  a  nurse 
for  blood  pressure  checks,  immunizations,  and 
throat  cultures.  Patients  were  also  identified 
because  of  multiple  telephone  consultations. 
After  some  initial  chart  review  we  used  only 
selected  CPT-4  codes  to  ensure  6  provider 
visits  in  primary  care  within  the  6  month 
window.   We  do  not  know  if  this  is  a 
limitation  to  our  system  or  to  the  medical 
billing  system  at  large. 


populations  identify  areas  for  quality 
improvement  among  primary  care  providers. 

In  this  era  of  health  care  reform  the 
quality  and  cost  of  medical  care  are  major 
focal  points.  Health  statistics  such  as 
anxiety  disorder  and  depression  screening 
rates  indicate  that  resources  should  be 
directed  to  the  recognition  and  treatment  of 
anxiety  disorder  and  depression  in  primary 
care.   These  statistics  can  also  be  used  to 
generate  interest  in  some  of  the  current  and 
future  public  health  needs  identified  by  the 
National  Institute  for  Mental  Health,  such  as 
the  prevention  of  mental  illness  and  promotion 
of  mental  health  in  the  primary  care  setting. 

As  the  process  of  developing  clinical 
practice  guidelines  evolves,  we  can  examine 
how  different  aspects  of  the  entire  care 
process  such  as  diagnosis,  treatment,  and 
management  fit  together.  This  information  can 
be  used  as  building  blocks  for  science-based 
performance  measurement  and  standard  setting 
in  health  care. 
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CONCLUSION 

In  conclusion,  measuring  the  screening  and 
diagnosis  rates  of  anxiety  disorder  and 
depression  in  primary  care  in  these  sub- 
populations  is  a  worthwhile  activity.  This 
activity  assesses  which  providers  are 
utilizing  practice  guidelines  for  screening 
and  diagnosing  these  mental  health  conditions. 
In  our  current  practice,  fatigue  and  sleep 
disorders  are  more  readily  identified  by 
primary  care  providers  as  being  risk  factors 
for  anxiety  disorder  and  depression.   The 
overall  screening  and  diagnosis  rates  for 
anxiety  disorder  and  depression  in  these 
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DISORDER 
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Continuity  of  care  following 
hospitalization  has  been  a  long-standing  concern 
in  the  mental  health  field.   Of  most  concern  is 
that  symptoms  of  depression  frequently  are  not 
recognized  by  practitioners  and  only  31%  of 
people  with  major  affective  disorders 
receive  treatment  of  any  kind  (Public  Health 
Service,  1990)  .  The  Health  Plan  Employer  Data 
and  Information  Set  (HEDIS  2.0)  has  drawn 
attention  to  this  concern  by  establishing  a 
quality  indicator  measuring  whether  a  patient 
was  seen  in  an  ambulatory  setting  within  30  days 
post  major  affective  disorder  discharge.  In  an 
effort  to  report  this  and  other  HEDIS  2.0 
measures  in  1994,  Blue  Choice,  an  Individual 
Practice  Association  (IPA)  model  HMO,  located  in 
Rochester,  New  York  with  more  than  410,000 
members  undertook  a  three  phase  analysis  of 
follow-up  care  rendered  to  patients  discharged 
with  the  diagnosis  of  major  affective  disorder. 
Phase  I  measured  compliance  with  the  HEDIS  2.0 
specifications.  Phase  II  measured  compliance 
with  the  NCQA  Report  Card  Pilot  Project 
specifications.  Phase  III  consisted  of  reviewing 
administrative  records  of  patients  not  meeting 
the  Phase  II  specifications  for  follow-up.   The 
Phase  I  specification  yielded  250  major 
affective  disorder  discharges  in  1993  (31%  of 
all  mental  health  discharges) .  The  30  day 
follow-up  rate  was  70.4  %.  The  phase  II  criteria 
increased  the  30  day  follow-up  rate  to  86.3%, 
with  a  denominator  of  204  discharges.  This  left 
28  patients  whose  discharge  aftercare  was 
undetermined.  Phase  III  demonstrated  that  out  of 
the  28  members,  20  members  had  been  followed  up 
within  30  days,  but  in  a  health  care  setting 
unrecognized  by  the  HEDIS  or  NCQA  Report  Card 
Pilot  Project  criteria.  The  types  of  follow-up 
care  rendered  are  summarized  and  discussed. 
These  findings  will  stimulate  further  discussion 
of  how  to  measure  quality  of  care  provided  to 
this  vulnerable  population. 


Introduction 

Major  affective  disorders,  particularly 
depression,  are  among  the  most  frequently 
occurring  psychiatric  disorders.   Up  to  5%  of 
U.S.  adults,  or  about  11  million  people,  may 
suffer  from  depression  at  any  one  time. 
According  to  a  new  study  by  a  team  of  economists 
at  the  Massachusetts  Institute  of  Technology, 
depression  costs  the  U.S.  $43.7  billion  per  year 
in  worker  absenteeism,  lost  productivity,  and 
health  care.      Depression  tends  to  be  more 
debilitating  than  diabetes,  arthritis, 
gastrointestinal  disorders,  back  problems,  and 
hypertension  in  terms  of  physical  functioning ,- 


interference  in  work,  housework,  or  schoolwork; 
and  normal  social  functioning'     As  a 
consequence,  the  Public  Health  Service,  in  its 
Healthy  People  2000  document,  has  established  a 
goal  to  increase  to  at  least  45%  the  proportion 
of  people  with  major  depression  disorders  who 
receive  treatment. 

One  concern  regarding  inpatient  treatment 
is  the  need  for  improved  continuity  of  care. 
Consistently,  aftercare  services  have  been  shown 
to  improve  the  quality  of  life  and  reduce  the 
clinical  symptoms  of  psychiatric  inpatients. 
Studies  suggest  that  recently  discharged 
inpatients  who  receive  some  kind  of  aftercare 
(e.g.,  medications,  psychotherapy,  occupational 
therapy,  or  case  management)  will  function 
better  after  leaving  the  hospital  than  those  who 
do  not.       However,  although  there  is 
widespread  recognition  of  the  need  for  providing 
transitional  care  services,  there  is  also  wide 
variability  in  the  extent  to  which  providers 
have  been  willing  to  take  responsibility  for, 
and  intensively  implement,  measures  to  ensure 
effective  aftercare.      This  is  one  reason  why 
ambulatory  follow-up  after  hospitalization  for 
major  affective  disorder  is  a  key  process 
measure  in  the  Health  Plan  Employer  Data  and 
Information  Set  (HEDIS  2.0).   This  paper  will 
describe  an  HMO's  first  year  experience  with  the 
HEDIS  2.0  Ambulatory  Follow-Up  after 
Hospitalization  for  Major  Affective  Disorder 
measure,  as  well  as  the  opportunities  for 
improvement  that  were  identified  both  in 
clinical  practice  and  in  the  HEDIS  2.0  measure 
specifications . 

This  study  encompasses  three  phases.   The 
first  phase  describes  our  1993  results  using  the 
HEDIS  2.0  specifications  for  the  Ambulatory 
Follow-Up  after  Hospitalization  for  Major 
Affective  Disorder  measure.   Phase  two  describes 
our  results  using  the  National  Committee  for 
Quality  Assures  (NCQA)  Report  Card  Pilot  Project 
specifications  for  the  same  measure,  and  phase 
three  reviews  what  happened,  and  illustrates 
ways  in  which  we  can  improve  care  provided  to 
our  members . 


Local  Environment 

r     In  Rochester,  N.Y.,  HMOs  enroll  54%  of  the 
insured  population,  the  highest  penetration  of 
managed  care  in  any  metropolitan  area  in  the 
United  States.   Rochester  is  unique  in  that  it 
has  a  long  history  of  effective  regional  health 
planning  and  capacity  control.   Hospital 
occupancy  rates  in  the  Rochester  area  average 
about  88%,  compared  with  a  national  average  of 
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68%.   In  Rochester,  instead  of  focusing  on 
competition  with  each  other,  local  hospitals 
work  together  to  coordinate  and  share  their 
services  to  avoid  unneeded  duplication. 

Blue  Choice  is  an  IPA-model  HMO 
established  in  1985,  and  operates  as  a  line  of 
business  of  Blue  Cross  and  Blue  Shield  of  the 
Rochester  (New  York)  Area.   With  over  410,000 
members  in  1993,  Blue  Choice  is  the  second 
largest  IPA-model  HMO  in  the  United  States. 

Blue  Choice's  mental  health  benefit 
structure  provides  full  coverage  for  30  days  of 
inpatient  hospitalization,  and  up  to  20 
outpatient  visits  with  a  50%  copayment .   The 
mental  health  panel  consists  of  psychiatrists, 
psychologists,  and  certified  social  workers. 
Services  rendered  by  non- participating  providers 
are  not  covered,  they  are  denied.   Blue  Choice 
also  has  a  Psychiatric  Case  Management  Program, 
Personalized  Benefit  Services  (PBS) ,  designed  to 
meet  the  needs  of  members  who  are  mentally  ill 
or  chemically  dependent.   The  goals  of  this 
program  are  to  provide  access  and  appropriate 
treatment  alternatives  to  members  while 
maintaining  high  quality,  cost-effective  care. 
PBS   case  managers  spot  opportunities  for 
alternative  care  that  may  be  more  appropriate, 
more  comfortable  for  the  patient  and  less 
expensive.   For  example,  PBS  case  managers  will 
use  what  is  remaining  in  the  mental  health 
inpatient  benefit  to  "buy"  ambulatory  care 
visits,  and  can  sometimes  utilize  the  chemical 
dependency  detoxification  benefit.   PBS  case 
managers  have  also  paid  for  travel  expenses  to 
get  patients  to  and  from  health  care 
appointments . 

Phase  I  Methods 

In  Phase  I,  the  HEDIS  2.0  specifications 
were  used  for  calculating  the  ambulatory  follow- 
up  after  hospitalization  for  major  affective 
disorder  rate.   All  hospital  admissions  for 
adult  members  ages  18-64  with  a  primary 
discharge  diagnosis  of  major  affective  disorder 
(i.e.,  ICD-9  code  of  296. xx)  were  identified. 

The  rate  was  then  calculated  based  on  the 
patients  in  the  target  population  who  received  a 
follow-up  visit  from  a  mental  health  provider 
within  30  days  after  their  discharge.   The 
follow  up  visit  must  have  been  coded  with  a 
major  affective  disorder  diagnosis.   The 
ambulatory  follow-up  visit  was  identified  using 
the  HEDIS  2.0  specified  CPT-4  and  revenue  codes 
as  well  as  local  codes  used  in  conjunction  with 
the  major  affective  disorder  ICD-9  diagnoses 
codes  (296 .xx) . 


CPT-4  Codes 

90801,  90841,  90843,  90844  & 

90844.22,  90847  &  90847.22, 

90849,90853,  90862,  90870, 

90871,  98900, and  98902 

Revenue 

900,  912,  913,  914,  915, 

Codes 

916,  918,  and  919 

Day  Hospital 

X9081,  X9082,  X9083,  and 

Local  Codes 

X9084 

Phase  I  Findings 

There  were  250  major  affective  disorder 
discharges  in  1993  (31%  of  all  mental  health 
discharges) .   Of  these  discharges,  176,  or 
70.4%,  had  received  ambulatory  follow-up  care 
within  30  days  of  discharge  (74  patients  had  no 
follow-up  care) . 

Phase  II  Methods 

In  Phase  II,  the  NCQA  Report  Card  Pilot 
Project  specifications  were  used  for  measuring 
the  ambulatory  follow-up  after  hospitalization 
for  major  affective  disorders.   The  target 
population  was  persons  who  were  18-64  years  of 
age  at  the  time  of  discharge,  who  had  been 
hospitalized  with  a  discharge  date  occurring 
during  the  first  330  days  of  the  reporting 
period,  had  a  principal  ICD-9  diagnosis  code 
indicating  a  major  affective  disorder  diagnosis 
(296. xx),  and  were   continuously  enrolled  for  30 
days   after  admission. 

The  ambulatory  follow-up  visit  was 
identified  using  only  the  HEDIS  2.0  specified 
CPT-4  codes,  revenue  codes  and  our  Plans'  local 
codes  (described  in  the  Phase  I  Methods) .   The 
rate  was  then  calculated  based  upon  the 
discharges  for  the  target  population  who 
received  a  follow-up  visit  with  a  mental  health 
provider  within  30  days,  with  no  limitation   on 
the  mental   health   diagnosis   for  this  visit. 

Phase  II  Findings 

Again  using  the  same  patient  base  as  in 
Phase  I,  there  were  204  major  affective 
disorder  patients  in  1993  who  met  the  age 
criteria  of  18-64  years  of  age  and  were 
continuously  enrolled  for  30  days  after 
admission.   (Please  note  that  Phase  I  (HEDIS 
2.0)  did  not  include  continuous  enrollment 
criteria,  and  therefore  the  denominator  has 
changed  from  250  in  Phase  I  to  204  in  Phase  II.) 
Of  the  204  major  affective  disorder  patients, 
176,  or  86.3%,  had  received  ambulatory  follow-up 
care  within  30  days  of  discharge.   This  left  28 
members  whose  aftercare  was  undetermined. 
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The  table  below  illustrates  the 
differences  between  Phase  I  (HEDIS  2.0)  and 
Phase  II  (NCQA  Pilot  Project)  results. 


#  of 
Discharges 

Ambulatory 
Follow-Up 

Undeter- 
mined 
aftercare 

HEDIS 
2.0 

250 

176 

74 

NCQA 

Pilot 

Project 

204 

176 

28 

Of  the  176  patients  that  received  follow- 
up  care,  152  or  86%  were  the  same  patients  in 
Phase  I  as  in  Phase  II .   The  24  patients  left 
were  either  not  included  in  Phase  I  due  to  the 
limitation  on  diagnosis  code  for  the  follow-up 
visit,  or  were  not  included  in  Phase  II  because 
of  the  continuous  enrollment  criteria. 

Phase  III  Methods 

Phase  III  research  consisted  of  reviewing 
each  of  the  remaining  28  members'  claims  history- 
online.   This  was  a  preliminary  step  in  an 
effort  to  set  up  a  process  for  ensuring  follow- 
up  within  30  days  for  future  major  affective 
disorder  discharges. 

Phase  III  Findings 

The  outcome  of  this  research  demonstrated 
that  20  of  the  28  patients  had  been  followed  up 
within  30  days,  but  in  a  health  care  setting  not 
specified  by  the  HEDIS  or  NCQA  Report  Card  Pilot 
Project  criteria. 

The  following  table  summarizes  the  way  in 
which  these  20  patients  received  follow-up  care: 


Reasons  for  Exclusion 

Number  of 
Patients 

Followed  up  by  a  non-mental 
health  provider 

-  Primary  Care  Physician 

-  Chemical  Dependency 
Provider 

7 

Followed  up  by  a  mental 
health  provider  but  were  not 
included  because  the  claims 
were  denied 

3 

Followed  up  but  were  not 
included  due  to  local  coding 

and  specification  issues 

10 

This  left  8  patients  who  did  not  receive 
follow-up  care  within  30  days  of  discharge.   The 
following  table  illustrates  what  happened  to 
those  eight  patients  who  did  not  receive  follow- 
up  care  within  30  days. 


Reasons  for  Exclusion 

Number  of 
Patients 

Received  follow-up  care 
after  30  days 

3 

Left  hospital  against 
medical  advice  and  no 
subsequent  follow-up 

1 

No  additional  mental  health 
utilization 

4 

Discussion 

It  has  been  demonstrated  that  seeking 
health  care  within  30  days  of  discharge,  for 
reasons  other  than  the  diagnosis  for  which  they 
were  admitted,  directly  correlates  to  the 
prevention  of  post-discharge  reactions. 
Therefore,  ambulatory  follow-up  visits  that  are 
not  diagnosis  driven  are  similarly  beneficial  in 
preventing  re-hospitalization.  (  '    The  NCQA 
Report  Care  Pilot  Project  specifications  did 
improve  our  plan's  measure  of  quality  due  to  the 
relaxed  specifications  on  mental  health 
diagnosis,  and  to  the  addition  of  the  continuous 
enrollment  criteria.   Using  these 
specifications,  our  plan's  rate  increased  23%, 
from  70.4%  to  86.3%. 

The  results  from  Phase  II  indicated  that 
we  did  have  some  missed  codes  in  which  patients 
should  have  been  in  our  follow-up  sample  (i.e., 
local  code  X919,  and  individual  case 
management) .    We  also  had  an  opportunity  to 
increase  our  follow-up  rate  by  using  our  denied 
claims  file,  as  our  analysis  revealed  3  patients 
who  did  receive  follow-up  care  but  whose  claims 
were  denied.   Once  these  gaps  are  closed,  we 
will  feel  more  comfortable  that  the  HEDIS 
specifications  represent  a  true  picture  of  the 
quality  of  care  provided. 


Interventions 

To  achieve  a  higher  ambulatory  follow-up 
rate  and  to  improve  mental  health  outcomes,  four 
interventions  are  being  developed  and 
implemented. 


58 


sai 


S«8 


4  . 


When  a  mental  health  patient  has  been 

discharged  from  an  inpatient  setting, 

our  referral  system  will 

automatically  generate  a  reminder  letter 

to  the  patient  about  their  first  follow-up 

visit . 

Fersonalized  Benefit  Services  case  managers 

will  begin  to  actively  participate  in 

discharge  planning  with  providers.   For 

example,  the  case  managers  will  make 

sure  that  the  patient  is  receiving  follow-up 

care  with  a  participating  provider,  so  that 

the  patients  will  receive  the  proper 

reimbursement.   As  stated  previously,  Blue 

Choice  will  not  reimburse  for  visits  to  a 

non-participating  provider. 

Personalized  Benefit  Services  case  managers 

will  also  make  sure  that  the  patient's 

follow-up  visit  is  occurring  within  a 

reasonable  amount  of  time  through  the 

discharge  planning  phase. 

Through  the  discharge  plan,  Personalized 

Benefit  Services'  case  managers  will  actively 

identify  potential  case  management  patients. 
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Recommendations 

After  thorough  analysis  of  the  28  members 
who  did  not  meet  the  HEDIS  2 . 0  or  NCQA  Report 
Card  Pilot  Project  criteria  for  follow-up  care, 
a  recommendation  was  made  to  NCQA  regarding  an 
improvement  to  the  Ambulatory  Follow-Up  after 
Hospitalization  for  Major  Affective  Disorder 
specifications . 


1.  Allow  visits  to  chemical  dependency  providers 
to  count  as  follow-up  care.   Blue  Choice, 
like  many  other  HMOs,  has  a  growing  mentally 
ill/chemical  abuser  (MICA)  population  in 
which  the  patient  is  served  by  both  systems. 
For  instance,  a  MICA  patient  will  be  admitted 
to  a  mental  health  inpatient  facility  to 
treat  his/her  mental  illness  and  when 
discharged,  that  patient  will  then  receive  an 
ambulatory  follow-up  visit  with  a  chemical 
dependency  provider  to  receive  care  for  their 
chemical  dependency. 

This  is  an  HMO's  first  effort  at  measuring 
and  improving  the  Ambulatory  Follow-Up  After 
Hospitalization  for  Major  Affective  Disorder 
rate.   Valuable  interventions  have  resulted  from 
this  research,  and  we  look  forward  to  continuing 
to  improve  the  quality  of  health  care  services. 
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THE  DELIBERATE  MISDIAGNOSIS  OF  MAJOR  DEPRESSION  IN  PRIMARY  CARE 

Carla  D.  Williams,  University  of  Arkansas  for  Medical  Sciences 

Kathryn  Rost 


Introduction 

Major  depression  is  a  prevalent  problem  in 
primary  care  settings.   Studies  estimate  that  5% 
-  10%  of  primary  care  patients  suffer  from 
depressive  illness.1  The  literature  suggests 
that  as  many  as  50%  of  patients  with  depressive 
symptoms  do  not  have  the  condition  diagnosed  by 
their  primary  care  physician.2 ,3   However, 
estimates  taken  from  existing  clinical  databases 
may  inaccurately  characterize  the  incidence  of 
depression  and  the  process  of  treatment  in  the 
primary  care  setting.   In  many  cases,  patients' 
psychiatric  symptoms  are  recognized  and 
sometimes  pharmacologically  treated,  however,  a 
psychiatric  diagnosis  is  not  noted  in  the 
patient's  record.11 

The  current  diagnostic  coding  system 
provides  physicians  with  an  array  of  codes  that 
can  be  substituted  for  depression.   Doctors 
often  deliberately  miscode  depressive  illness  to 
avoid  many  of  the  negative  consequences  of 
making  a  psychiatric  diagnosis.5   For  primary 
care  physicians,  a  major  consequence  of 
diagnosing  and  treating   depression  is  being 
subjected  to  lower  rates  of  reimbursement  by 
third-party  payers.6   Tolerating  inequitable 
reimbursement  policies  is  an  undesirable  outcome 
for  providing  appropriate  care  and  may  provide 
an  incentive  for  using  alternative  diagnostic 
codes . 7 

It  is  important  to  note,  however,  that 
many  of  the  reasons  for  alternative  coding  are 
not  related  to  unacceptable  outcomes  for 
physicians,  but  rather  to  the  negative  impact  of 
subsequent  stigmatization  of  patients  who 
receive  a  psychiatric  diagnosis.   Many 
physicians  are  aware  that  a  psychiatric 
diagnosis  can  delay  recovery  and  jeopardize  a 
patient's  future  ability  to  obtain  life  and 
health  insurance,  to  secure  disability  benefits, 
and  to  gain  employment.   Physicians  are 
therefore  cautious  in  making  a  diagnosis  of 
depression  for  fear  of  harmful  repercussions  for 
their  patients. 

Although  primary  care  physicians  may  seem 
to  have  justifiable  reasons  for  deliberately 
miscoding  major  depression,  the  practice  of 
deliberate  misdiagnosis  invalidates  the  use  of 
existing  databases  in  determining  the  actual 
role  of  primary  care  physicians  in  the  provision 
of  mental  health  treatment .   Under  current 
health  care  policy,  insurance  coverage  and 
reimbursement  rates  for  psychiatric  disorders 
are  often  subject  to  more  limitations  than 
coverage  and  reimbursement  for  physical  health 
problems.8   The  decisions  which  allow  such 
inequities  to  continue  are  based,  in  part,  upon 
inaccurate  data  which  indicate  psychiatric 
disorders  are  infrequently  or  ineffectively 
treated  in  primary  care  settings. 

To  address  problems  with  utilizing  current 
clinical  databases  to  characterize  the  delivery 
of  mental  health  services  in  primary  care 
settings,  a  study  was  undertaken  to  estimate  the 
prevalence  of  alternative  coding  of  depression 
in  primary  care  and  to  identify  the  reasons  for 
its  occurrence. 

Methodology 

Primary  care  physicians  from  practices 
across  the  U.S.  were  randomly  selected  from  the 
membership  directories  of  the  American  Academy 
of  Family  Physicians  and  the  American  College  of 
Physicians.   Equal  numbers  of  physicians  from 
each  professional  group  were  selected  to  yield  a 
total  sample  of  634  physicians. 


A  one-page  questionnaire  was  mailed  to 
each  participant.   After  two  weeks,  a  second 
survey  was  mailed  to  physicians  who  had  not 
responded.   Physicians  who  did  not  respond  to 
either  mail  survey  were  contacted  by  telephone  3 
-  5  weeks  after  the  initial  mailing.   To 
maximize  the  response  rate,  extensive  socio- 
demographic  data  and  information  about  practice 
characteristics  were  not  collected  from 
individual  physicians.   The  professional 
organizations  that  supplied  physician  names  and 
addresses  identified  physician  specialty. 
Information  about  the  demographic 
characteristics  of  the  county  in  which  the 
physician  practiced  was  obtained  from  the  1990 
U.S.  Census. 

The  survey  asked  respondents  to  estimate 
the  number  of  adult  outpatients  they  had  seen  in 
the  preceding  two  weeks  and  to  estimate  how  many 
of  those  patients  met  DSM-III-R  criteria  for 
major  depression  at  the  time  of  the  visit.   (A 
one-page  guideline  for  diagnosing  major 
depression  was  included  on  the  reverse  side  of 
the  questionnaire.) 

Physicians  were  asked  if  they  used  an 
alternative  diagnostic  code  for  major  depression 
with  any  of  their  patients.   If  so,  they  were 
asked  to  estimate  how  many  of  those  patients 
received  an  alternative  code.   Doctors  who 
reported  deliberately  miscoding  major  depression 
were  given  a  list  of  nine  possible  alternate 
codes  and  asked  to  identify  the  three  codes  most 
commonly  substituted.   Additionally, 
participants  were  given  nine  reasons  for 
deliberate  miscoding  and  were  asked  to  list  the 
three  most  compelling  reasons  for  using 
alternative  codes. 

When  physicians  were  given  multiple 
response  options,  an  "other"  category  was 
included.   Those  who  selected  "other"  were  asked 
to  specify  an  alternate  response.   The  content 
of  those  responses  was  analyzed  and,  where 
appropriate,  incorporated  into  closely  related 
existing  categories.   Two  new  response 
categories  for  reasons  for  miscoding  were 
developed  as  a  result  of  the  content  analysis. 

Results 

Surveys  were  completed  for  70%  (N  =  444) 
of  participants.   Sixty-four  percent  of 
completed  surveys  were  returned  by  mail  and  36% 
were  completed  by  telephone  interview.   Twenty- 
two  percent  (N  =  138)  of  physicians  sampled 
could  not  be  located  by  mail  or  telephone  and  8% 
(N  =  52)  refused  to  participate. 

Physicians  estimated  that  6.6%  of  patients 
seen  during  the  preceding  two  week  period  met 
DSM-III-R  criteria  for  major  depression. 
Analysis  of  responses  revealed  that  50.3%  of 
physicians  (N  =  192,  SE,  +2.5%)  reported  using 
an  alternative  code  for  at  least  one  patient 
with  major  depression  during  the  past  two  weeks. 
Of  patients  judged  to  meet  DSM-III-R  criteria 
for  major  depression,  31%  received  an  alternate 
diagnostic  code. 

Table  1  presents  the  proportion  of 
physicians  who  reported  using  each  of  the  nine 
possible  alternative  diagnostic  codes. 
Fatigue/malaise,  insomnia,  and  headache  were  the 
three  most  frequently  substituted  codes. 
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Table  1. 

Diagnoses  Most 

Frequently 

Substituted" 

Fatigue /malaise 

Insomnia 

Headache 

Anxiety 

Adjustment/grief  reaction 

Other 

Fibromyalgia 

Anorexia 

Premenstrual    syndrome 

Irritable   bowel   syndrome 


Respondents   could  check  up   to   three   diagnoses. 


%   of    Physicians 
Using  Diagnosis 


59.8 
43  .9 


28 

25 

23 

20 

19 

14.8 
4.8 
4.2 


Uncertainty  about  the  diagnosis  was  the 
reason  most  frequently  cited  for  alternative 
coding  of  major  depression.   Difficulty  with 
reimbursement  for  depression  treatment  was  the 
second  most  common  reason  for  substitution. 
Table  2  lists  the  proportion  of  respondents 
citing  each  reason  for  alternative  coding.   As  a 
result  of  the  content  analysis  of  responses 
written  in  by  physicians,  two  additional 
categories  were  identified:   Patient's 
resistance  to  the  diagnosis  and  physician's 
preference  toward  selecting  diagnostic  codes 
that  reflect  specific  symptoms. 


Table  2. 

Most  Frequently 
Cited  Reasons 
for  Substitution 

Uncertainty  about 
the  diagnosis 

Problems  with 

reimbursement  for 
services  if 
depression  is  coded 

Jeopardize  future 
ability  to  obtain 
health  insurance 

Stigma  associated  with 
depression  will  delay 
patient's  recovery 

Jeopardize  future 
ability  to  obtain 
life  insurance 

Stigma  associated  with 

depression  will  negatively 
influence  future  care  from 
other  providers 

Patient  unwilling  to 
accept  diagnosis 

Explicit  request 
from  patient 

Jeopardize  future  employment 

Other 

Jeopardize  future  ability 
to  obtain  disability 

Prefer  codes  that  reflect 
specific  symptoms  of 
presenting  complaint 


%  of  Respondents 
Citing  the  Reason 


46  .  0 

44.4 
29.4 
20.9 
12.8 

12.3 

11.8 

11.2 

10.2 

8.6 

6.4 
6.4 


Internists  were  more  likely  to  report 
using  alternative  codes,  56.1%  (96/171)  than 
family  physicians,  45.5%  (96/211)  (x2  =  4.28,  p 
=  .04) .   Physicians  who  reported  using 
alternative  codes  for  major  depression  estimated 
seeing  10.4  depressed  patients  during  the 
preceding  two  week  period  while  physicians  who 
reported  they  did  not  use  alternative  codes 
estimated  seeing  6.0  depressed  patients  (t  = 
5.19,  p  =  .0001) .   The  volume  of  patients  seen 
in  the  practice  did  not  predict  use  of 
alternative  codes. 

The  following  demographic  characteristics 
for  the  counties  where  physicians  practiced  were 
examined:   the  proportion  of  the  population  age 
2  5  and  over  with  a  high  school  education;  the 
proportion  of  the  population  age  65  and  over; 
the  proportion  of  ethnic  minorities;  and  median 
family  income.   Physicians'  use  of  alternative 
codes  did  not  differ  based  on  any  of  the 
aforementioned  characteristics  of  county  of 
practice  location. 

Discussion 

Results  of  the  study  demonstrate  that 
deliberate  misdiagnosis  of  major  depression  is  a 
common  practice  among  primary  care  physicians. 
Half  of  all  physicians  surveyed  reported  using 
alternative  diagnostic  codes  for  major 
depression.   One  third  of  primary  care  patients 
with  major  depression  received  an  alternate 
code.   These  findings  indicate  that  current 
clinical  databases  may  grossly  underestimate  the 
prevalence  of  major  depression  in  the  primary 
care  setting.   Moreover,  the  practice  of 
deliberate  misdiagnosis  precludes  any  realistic 
estimation  of  the  primary  care  physician's  role 
in  the  diagnosis  and  treatment  of  major 
depression. 

The  reason  most  frequently  cited  for  using 
alternative  codes  was  the  physician's 
uncertainty  about  the  diagnosis.   Forty-six 
percent  of  respondents  said  diagnostic 
uncertainty  was  a  compelling  reason  for 
alternative  coding.   This  indicates  that  in 
order  to  more  accurately  characterize  the 
treatment  of  depressive  disorders  in  primary 
care,  it  will  be  necessary  to  increase 
physicians'  confidence  in  making  the  diagnosis. 

Physicians  obviously  recognize  a 
substantial  number  of  their  patients  who  meet 
criteria  for  major  depression.   However,  doctors 
often  lack  confidence  in  their  ability  to 
adequately  treat  patients  with  major  depression 
while  avoiding  many  of  the  negative  consequences 
associated  with  being  diagnosed  with  a 
depressive  illness.   Providing  guidelines  for 
the  diagnosis  and  treatment  of  psychiatric 
disorders  in  primary  care  may  reduce  some  of  the 
diagnostic  uncertainty  and  thereby  alleviate  the 
reticence  physicians  exhibit  in  treating  their 
depressed  patients. 

Future  Data  Needs 

Data  obtained  from  detailed  reviews  of 
written  medical  records  could  provide  important 
insight  into  physician  recognition  of  depression 
and  the  consequent  treatment  decisions. 
Systematic  reviews  of  physicians'  notations 
could  potentially  reveal  awareness  of  patients' 
depression  in  the  face  of  diagnostic 
uncertainty.   Comparisons  of  physicians' 
detailed  patient  notes  to  the  diagnostic  codes 
actually  recorded  would  identify  instances  where 
depression  was  a  potential  diagnosis,  but 
alternative  codes  were  recorded. 

Likewise,  creating  and  maintaining 
computer  databases  that  contain  patients' 
complete  problem  lists  could  be  beneficial  in 
identifying  patients  with  depressive 
symptomatology.   The  utility  of  computer- 
assisted  diagnostic  coding  based  on  patient 
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problem  lists  is  already  being  explored.9   It 
is  reasonable  to  believe  that  the  recording  of 
psychiatric  disorders  presenting  to  the  primary 
care  setting  could  be  improved  through  the  use 
of  computer-aided  coding  technology. 

Additionally,  enhancing  interdisciplinary 
treatment  networks  could  serve  to  bolster 
primary  care  physicians'  willingness  to  address 
mental  health  problems  with  their  patients. 
Availability  of  consultation  and  referral 
services  would  provide  networks  of  support  for 
physicians  who  encounter  difficulties  in 
treating  depressed  patients.   Maintaining  data 
on  formal  and  informal  physician  referrals  to 
specialty  mental  health  care  would  provide 
another  indicator  of  the  recognition  of 
depression  and  other  psychiatric  disorders  in 
the  primary  care  setting. 

Conclusions 

While  it  is  important  to  address 
physicians'  concerns  about  adequately  treating 
depressed  patients  in  the  primary  care  setting, 
the  crux  of  the  problem  of  deliberate 
misdiagnosis  lies  with  the  second  most  common 
reason  physicians  gave  for  deliberately 
misdiagnosing  major  depression.   Forty- four 
percent  of  physicians  noted  problems  with 
reimbursement  for  treating  psychiatric  disorders 
as  a  compelling  reason  for  employing  deliberate 
miscoding . 

This  study  reveals  that  deliberately 
miscoding  major  depression  is  a  tactic  widely 
used  by  physicians  to  circumvent  the  negative 
consequences  of  treating  mental  health  problems 
in  the  primary  care  setting.   Physicians' 
decisions  to  employ  deception  to  avoid  the 
restrictions  third  party  payers  place  upon 
reimbursement  for  mental  health  treatment  serves 
only  to  perpetuate  the  inequitable  policies  now 
in  existence.   Allowing  insurers  to  continue  to 
enforce  discriminatory  policies  does  a  great 
disservice  to  both  doctor  and  patient. 

Inequitable  coverage  for  mental  health 
problems  is  based  upon  the  idea  that  treatment 
for  mental  health  problems  is  less  beneficial 
than  treatment  for  medical  conditions.   There  is 
a  preponderance  of  evidence  to  support  the 
contrary.   Many  effective  treatments  for  major 
depression  are  now  available.10   Physicians 
have  a  responsibility  to  challenge  these  ill- 
founded  policies  by  correctly  identifying  and 
treating  depressed  patients  in  the  primary  care 
setting  and  reporting  the  actual  process  and 
outcomes  of  that  treatment.   Analysis  of 
clinical  databases  would  then  provide  clear 
evidence  of  the  primary  care  physician's  role  in 
treating  depressive  disorders.   As  the  actual 
rates  of  treatment  of  depression  in  primary  care 
become  apparent  and  the  good  outcomes  of  those 
treatments  are  demonstrated,  insurers  would  be 
forced  to  re-examine  the  policies  that 
potentially  limit  a  patient's  ability  to  receive 
appropriate  treatment  in  a  low  cost  setting  such 
as  primary  care. 

In  the  mean  time,  it  is  necessary  to 
collect  primary  data  which  accurately  describes 
the  prevalence  of  depressive  illness  among 
primary  care  patients.   This  can  be  accomplished 
by  utilizing  one  of  several  screening  tools 
normed  on  primary  care  populations.11  12 
Additionally,  it  is  necessary  to  collect  and 
disseminate  information  about  the  outcomes  of 
treatment  for  depression  received  from  the 
primary  care  sector.   An  Outcomes  Management 
System  has  been  developed  explicitly  for  the 
purpose  of  monitoring  the  outcomes  of  depression 
treatment.13   Data  collection  and  dissemination 
by  researchers  will  support  the  primary  care 
physician's  efforts  to  dispel  the  myths  about 
depression  among  primary  care  patients.   In 
partnership  with  researchers,  primary  care 
physicians  can  effectively  demonstrate  the 
frequency  with  which  depressed  patients  seek 


treatment  from  primary  care  providers  and  can 
allow  for  fact-based  comparisons  of  treatment 
outcomes  in  primary  and  specialty  care  settings. 
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DEVELOPING  AND  TESTING  DATA  COLLECTION  INSTRUMENTS  TO  CHARACTERIZE  PSYCHIATRIC 
PATIENTS  AND  CLINICAL  PRACTICE  PATTERNS 

Joyce  C.  West,  American  Psychiatric  Association 

Purpose:   The  APA  is  developing  a  Practice  Re- 
search Network  (PRN)  to  conduct  clinical  and 
services  research.   An  initial  objective  is  to 
develop  core  instruments  to  systematically 
characterize  Network  members,  their  patients, 
and  clinical  practice  patterns. 

Methods:      The  core  Network  member  instrument 
collects  data  on  Network  members'  demographics, 
training,  and  practices.   The  core  patient- 
level  instrument  collects  data  on:   patient 
sociodemographics;  treatment  setting/  plan; 
diagnostic  information  (all  DSM-IV  axes) ;  and 
mental  health  treatments.   One  hundred  seventy- 
six  APA  members  practicing  in  the  full  range  of 
inpatient  and  outpatient  treatment  settings 
volunteered  to  participate  in  the  Network  and 
test  these  instruments.   The  instruments  and 
protocols  were  mailed.   Reminder  postcards, 
telephone  and  fax  reminders  were  administered. 

Findings:      Ninety  percent  of  the  17  8  Network 
members  completed  the  Network  member  survey; 
95%  of  the  158  members  who  responded  to  the 
survey  completed  the  patient-level  instrument 
on  five  systematically  selected  patients,  pro- 
viding data  on  300  patients.    Quality  of  the 
data  appears  good  based  on  the  low  missing  data 
rate  and  initial  validation.   Data  on  the 
sociodemographic  characteristics  and  practice 
settings  of  Network  members  compared  to  psychi- 
atrists sampled  in  other  national  data  bases 
will  be  presented.   The  majority  of  patients 
had  psychiatric  comorbidities:   57%  had  one  or 
more  Axis  I  mental  disorders  in  addition  to  the 
principal  disorder;  21%  had  two  or  more. 
Thirty-nine  percent  had  one  or  more  Axis  II 
personality  disorders  in  addition  to  the  prin- 
cipal disorder;  41%  had  one  or  more  Axis  III 
general  medical  disorders.   The  most  common 
treatments  were  psychiatric/medical  management 
(65%),  individual  psychotherapy  (53%),  and  med- 
ication prescriptions/monitoring  (51%).   Ninety 
percent  of  the  patients  received  at  least  one 
psychotropic  medication;  the  most  common  agent, 
fluoxetine,  was  prescribed  for  22%  of  the  pa- 
tients.  A  significant  proportion  of  patients 
received  mental  health  treatments  from  other 
providers;  for  example,  24%  of  outpatients  re- 
ceived individual  psychotherapy  from  another 
provider . 

Conclusions :      These  pilot  studies  demonstrate 
the  feasibility  of   the  PRN  in  collecting  de- 
tailed patient-level  data  from  large  numbers  of 
psychiatrists.   Outstanding  methodologic  is- 
sues, including  sampling  and  analytic  plans  and 
instrument  validation,  will  be  discussed. 
Plans  are  underway  to  expand  the  Network  and 
conduct  a  variety  of  studies,  including  longi- 
tudinal clinical  effectiveness  studies. 
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Introduction 

The  President's  Childhood  Immunization 
Initiative  establishes  a  national  priority  for 
the  immunization  of  preschool  children.   It 
sets  the  goal  of  achieving  immunization  levels 
of  90%  for  the  routinely  recommended  childhood 
vaccines  in  two-year-old  children  in  this 
country  by  the  year  2000.   Immunizations  are  a 
cost-effective  way  to  prevent  childhood 
diseases  and  assure  a  healthy  pediatric 
population1.  By  preventing  illness, 
immunizations  save  lives,  prevent  suffering, 
and  reduce  medical  costs2.   Clearly,  efforts 
should  be  made  to  maximize  the  number  of 
children  immunized. 

A  unified  schedule  for  childhood 
immunizations  was  disseminated  for  use  in 
January  19953.  In  accordance  with  the 
schedule,  the  majority  of  primary 
immunizations  should  be  received  by  age  two. 
However,  many  children  do  not  receive  their 
immunizations  as  scheduled.  Because  of  this 
statistic,  the  responsibility  for  achieving 
high  immunizations  rates  must  be  shared  by 
many  people  including  parents,  health  care 
providers,  and  public  health  agencies4. 

In  1993,  the  Minnesota  Department  of 
Health  estimated  that  only  61.4%  of  the 
state's  two-year-olds  had  completed  the  basic 
primary  series  on  schedule5.  There  is 
currently  no  centralized  system  to  monitor 
childhood  immurization  status.  The  only  way  to 
gather  this  information  was  to  do  a 
retrospective  review  of  the  records  of 
children  enrolled  in  kindergarten  and  record 
the  number  of  immunizations  listed  as  received 
by  age  two . 

Mayo  Clinic,  Rochester  Minnesota, 
provides  primary  care  for  individuals  and 
families  in  the  surrounding  geographical  area. 
At  Mayo  Clinic  Rochester,  primary  care  for 
children  is  provided  by  two  Departments: 
Community  Pediatrics  and  the  Rochester  Family 
Practice  Clinic.  Pediatric  patients  can  also 
receive  medical  care  at  an  Urgent  Care  Center 
(UCC) .  The  Kasson  Mayo  Family  Practice  Clinic 
is  located  15  miles  west  of  Rochester  in 
Kasson  Minnesota.  This  clinic  provides  primary 
care  within  a  rural  setting. 

The  Mayo  Clinic  and  other  health  care 
organizations  from  the  Midwest  have 
collaboratively  joined  to  address  Continuous 
Quality  Improvement  in  healthcare.  These 
organizations  develop  clinical  practice 
guidelines  to  assist   the  clinician  in  the 
delivery  of  high  quality,  clinically 
appropriate,  cost-effective  health  care  to  all 
segments  of  the  population  in  accordance  with 
economically-efficient  practices  based  on  the 
principles  of  continuous  improvement6.  Each 
guideline  focuses  on  a  very  specific  aspect  of 
health  care,  such  as  Pediatric  Immunizations 


and  each  includes  measurement  specifications. 
The  measurement  results  are  reported  back  to 
the  individual  medical  groups  to  serve  as  a 
basis  for  Continuous  Quality  Improvement. 
Each  group  can  view  its  progress  in  reducing 
variation  and  producing  better  outcomes. 

A  Pediatric  Immunization  Health  Care 
Guideline  was  implemented  at  Mayo  in  1994. 
The  results  of  the  assessment  performed  to 
determine  adequacy  of  pediatric  immunization 
rates  for  1994  are  presented  in  this  paper. 

Methods 

The  population  under  study  consisted  of 
children  seen  in  each  of  the  two  family 
practice  clinics,  the  Community  Pediatrics 
practice  or  the  Urgent  Care  Center  during 
1994.  Patients  were  identified  through  our 
ambulatory  services  billing  system.  Children 
were  included  if  between  the  ages  of  two  and 
two  and  one-half-years-old  at  the  time  of 
their  visit.   The  purpose  of  the  visit  could 
have  been  for  either  acute  or  maintenance 
health  care.  Random  samples  of  10  patients  per 
month  per  site  were  selected  for  a  total 
sample  of  480  patients.  Medical  records  were 
then  reviewed  to  determine  immunization  status 
by  the  child's  second  birthday  for  diphtheria, 
pertussis,  tetanus  (DPT);  polio;  measles, 
mumps,  rubella  (MMR) ;  and  Haemophilus 
influenzae  type  b  (HIB) .   Hepatitis  B 
immunization  is  currently  not  required  for 
school  or  daycare  attendance  in  MN.,  so  data 
regarding  it  is  not  included.  Within  the 
guideline,  there  is  a  schedule  for 
administration  of  Hepatitis  B  vaccine  either 
during  infancy  or  during  adolescence.  Well- 
child  care  received  within  the  Mayo  System  was 
also  recorded. 

Credit  was  only  given  for  an  immunization 
in  which  the  date  (month/year)  was  recorded  in 
the  child's  record.  The  immunization  could 
have  been  given  within  the  Mayo  System  or 
elsewhere.  Documentation  of  immunizations 
being  "up  to  date"  was  not  considered 
sufficient .  The  number  of  immunizations 
required  to  be  considered  complete  by  age  2 
are  presented  in  Table  1 . 


Table  1.  Required  Immunizations 

4  Diphtheria,  Pertussis,  Tetanus  (DPT) 

3  Polio  (oral  or  inactivated) 

1  Measles,  mumps,  rubella  (MMR) 

4  Haemophilus  influenza  type  b  (Hib) 


67 


Due  to  the  sampling  scheme,  overall 
immunization  rates  were  weighted  by  the  number 
of  children  seen  at  each  site.   Statistical 
comparisons  between  sites  were  based  on  Chi- 
square  analysis. 

Results 

The  overall  immunization  rate  by  age  two 
for  all  four  sites  in  1994  was  84%.  This 
immunization  rate  is  not  population-based,  but 
reflects  the  rate  among  two  to  two  and  one- 
half-year-olds  brought  to  one  of  the  four 
sites.  The  overall  rate  has  been  adjusted  to 
reflect  the  population  seen  at  each  site, 
since  fewer  two-year-olds  are  seen  at  the 
Kasson  Family  Practice  site  than  at  Community 
Pediatrics.   The  completeness  rate  for  each  of 
the  four  sites  is  presented  in  Table  2. 

Table  2 .   Completeness  Rate  by  Site 


Site 

Community  Pediatrics 
Family  Medicine  Rochester 
Urgent  Care  Center 
Rural  Family  Medicine 


91 
87 
83 
48 


record  regarding  their  immunizations  and  22% 
had  missed  one  dose  of  the  series  (Figure  1) . 

The  type  of  health  care  visit  history 
for  these  children  was  examined.   97%  of  the 
children  had  well-child  care  visits  at  the 
three  Rochester  based  sites.  Only  75%  of  the 
children  had  well-child  care  visits  at  the 
Kasson  Family  Practice;  25%  of  children  under 
age  two  were  seen  for  acute  care  only  (Figure 
2)  . 

Immunization  rates  were  affected  by  more 
than  well-child  care  rates  alone.   As  shown  in 
Figure  3,  children  who  received  well-child 
care  at  the  Rochester-based  sites  had  an 
immunization  rate  of  87%,  while  the  rate  at 
the  Rural  Family  Practice  site  was  61%.   The 
immunization  rates  for  children  who  only  had 
acute  care  visits  were  similarly  low  at  all 
sites . 

The  immunization  rate  at  the  UCC  was 
83%.   96%  of  the  two-year-olds  seen  at  the  UCC 
during  1994  had  received  well-child  care 
within  the  Mayo  System  either  at  Community 
Pediatrics  or  at  Family  Medicine.  The  high 
utilization  of  the  UCC  by  children  ages  two  to 
two  and  a  half  who  also  received  well-child 
care  within  Mayo  and  low  utilization  by  the 
Rochester  community  at  large  led  to  the 
unexpected  high  immunization  rate  in  the  UCC. 

Discussion 


Two  findings  were  unexpected:  the 
completeness  rate  for  the  Kasson  Family 
Practice  site  is  significantly  lower  than  the 
Rochester-based  sites  (p<0.001),  and  the  rate 
at  the  UCC  was  higher  than  anticipated. 

Since  the  immunization  rate  at  the 
Kasson  Family  Practice  site  was   39%  lower 
than  the  average  rates  for  the  other  three 
sites,  potential  underlying  reasons  were 
investigated.  The  rate  by  type  of  immunization 
comparing  the  Kasson  practice  to  the  overall 
completeness  rate  for  all  4  sites  is  presented 
in  Table  3.  With  the  exception  of  MMR,  which 
is  only  given  once,  the  other  three  rates  were 
similar  at  Kasson. 


Table  3 .   Completeness  Rate  by  Type  of 
Immunization 


Rural  Family 

All 

4  si 

tes 

Medicine 

DPT 

86 

52 

Polio 

88 

57 

MMR 

94 

70 

Hib 

91 

57 

Next,  it  was  determined  whether  the 
children  seen  at  the  Kasson  Family  Practice 
lacked  documentation  of  any  immunizations 
being  given,  or  if  the  primary  series  had  been 
started,  but  not  completed.   44%  of  the 
children  had  not  completed  all  three  series. 
On  average  for  the  three  series,  19%  of  the 
children  had  no  documentation  in  their  medical 


The  process  of  implementing  the  Pediatric 
Immunization  Guideline   differed  between  the 
Rochester-based  sites  and  the  Kasson  Family 
Practice  Clinic.   As  part  of  the 
implementation  process  at  both  Community 
Pediatrics  and  Rochester  Family  Practice, 
steps  were  taken  to  assure  high  immunization 
rates.  They  chose  to  identify  those 
preschoolers  who  did  not  have  documented 
evidence  that  they  have  completed  their 
primary  series  of  immunizations  and  to  either 
immunize  them  or  document  immunizations 
received  elsewhere. 

As  an  appointment  is  made  for   a  child 
between  the  ages  of  15  months  and  5  years  of 
age  at  the  Rochester  Family  Practice  Clinic  or 
Community  Pediatrics  Practice,  there  is  a 
verbal  reminder  to  bring  their  immunization 
card  to  that  appointment.  A  nurse  or  physician 
then  checks  their  card  against  their  record  to 
assess  completeness  of  immunizations.  This  is 
done  regardless  of  type  of  appointment.  If  the 
primary  series  of  immunizations  is  complete, 
the  record  is  stamped  with  a  red  stamp  that 
says  "Immunizations  Complete  by  Age  2.'  If  not 
complete,  the  necessary  immunization  is  given 
that  day  or  a  follow-up  appointment  is  made. 

Medical  records  are  also  screened  for 
immunization  status  when  being  used  for 
reasons  that  do  not  involve  a  patient  visit, 
such  as  for  dictation.  If  there  is  evidence 
that  immunizations  are  complete,  the  record  is 
stamped  complete.  If  not  complete,  a  letter  is 
sent  to  the  child's  home  stating  that  the 
medical  record  indicates  that  the  child's 
immunizations  are  delayed.  They  are  requested 
to  call  with  the  dates  if  immunizations  were 
received  elsewhere  or  to  make  an  appointment 
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to  get  the  necessary  immunizations.  If  there 
no  response  to  the  letter  within  two  weeks, 
one  reminder  phone  call  is  made. 

The  Kasson  Family  Practice  did  not  have 
such  a  process  in  place  in  1994.  In  addition, 
fewer  of  the  children  seen  at  the  Kasson 
Family  Practice  site  were  receiving  routine 
well-child  care.  Without  a  source  of  regular 
care,  children  have  little  opportunity  to 
receive  the  full  schedule  of  vaccines,  much 
less  on  time.   A  child  stands  a  far  better 
chance  of  being  fully  immunized  if  he  or  she 
has  one  primary  care  provider  who  is  likely  to 
monitor  the  child's  immunization  status  on  a 
regular  basis  and  to  provide  the  required 
immunizations  on  site  when  these  needs  are 
identified7 . 

As  part  of  the  Continuous  Quality 
Improvement  Process,  the  results  were 
presented  to  the  Kasson  Family  Medicine 
providers .  They  have   reviewed  the  process 
they  use  when  seeing  young  children  and  plan 
to  apply  some  of  the  steps  utilized  by  the 
Rochester-based  groups.  Since  then,  their 
first  quarter  1995  immunization  data  has  been 
reviewed  and  the  trend  is  positive.  Their 
overall  completion  rate  increased  from  48%  in 
1994  to  63%.  A  medical  record  review  will  be 
continued  on  a  quarterly  basis,  so  as  to 
continue  to  provide  feedback  to  all  the 
medical  groups . 

Conclusions 

•  Immunization  rates  at  Rochester-based  sites 
were  higher  than  those  at  the  Rural  Family 
Practice . 

•  Children  who  only  receive  acute  care  have 
lower  immunization  rates  than  those 


receiving  well-child  care. 

•  In  addition,  an  opportunity  for  improvement 
in  immunization  rates  exists  among  children 
who  receive  well-child  care  at  Rural  !■■ 
Practice . 

•  Different  implementation  processes  were 
used  at  various  sites. 

•  There  is  a  need  to  utilize  the  acute  care 
visit  to  review  immunization  status. 

Each  health  care  encounter  is  an 
opportunity  to  immunize.  The  practitioner 
should  use  every  visit  to  check  immunization 
records,  give  needed  vaccines,  educate  parents 
about  immunization,  and  schedule  the  next 
immunization  appointment. 
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Figure  1:   Rural  Family  Medicine  Performance  for  DPT,  Polio,  HIB 
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Figure    2:       Well-Child   Care 
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Figure  3:   Immunization  Rate  by  Type  of  Care 
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IMMUNIZATION  RATES  FOR  A  HOUSTON  HMO 


Dorothy  McCraw  Syblik,  NYLCare/Sanus  Healthplan 


INTRODUCTION 

A  Health  Maintenance  Organization  (HMO)  in 
Houston,  Texas  conducted  a  mail  survey  of 
parents  of  children  who  were  one  year  old  in 
1993.  A  child  was  classified  as  immunized  if 
he/she  had  3  DTPs,  2  OPVs,  and  3  Hibs.  1297 
surveys  were  mailed  and  there  was  no  additional 
followup.  There  was  a  24%  response  rate.  The 
immunization  rate  was  89%,  much  higher  than  pre- 
viously reported  for  Houston. 

In  Houston,  articles  in  the  local  newspapers 
stated,  "CDC  reports  Houston  to  have  one  of  the 
lowest  immunization  rates  in  the  nation." 

The  Center  for  Disease  Control  conducted  a 
retrospective  analysis  of  school  age  children 
for  vaccination  levels  at  their  second  birthday 
for  nine  cities  (1).  The  proportion  of  children 
who  were  up-to-date  with  valid  doses  by  their 
second  birthday,  based  on  the  4  DTP:3  OPVtl  MMR 
schedule  was  10%  in  Houston.  When  up-to-date 
vaccination  levels  were  determined  without 
application  of  the  strict  definitions  for  timing 
of  valid  doses,  the  coverage  rates  increased  by 
1%  to  6%. 

The  mail  survey  results  were  suspicious  even 
though  the  different  study  designs  account  for 
some  of  the  difference.  There  is  some  evidence 
that  the  HMO's  have  higher  immunization  rates 
than  the  community  as  a  whole.  A  study  of  six 
HMO's  in  the  Dallas  Fort  Worth  Area  reported  up- 
to-date  immunizations  in  two  year  old  children 
in  1993  and  found  immunization  rates  to  be  47% 
(2).  A  similar  study  of  six  HMO's  from  the 
report  card  project  reported  average  immuniza- 
tion rates  at  79%  for  two  year  old  children  in 
1993  (3). 

The  Houston  HMO  study  reported  one  of  the 
highest  HMO  immunization  rates  in  a  community 
with  one  of  the  lowest  reported  rates. 

How  valid  and  reliable  is  the  mail  survey 
instrument?  A  validation  study  by  medical  record 
review  of  immunizations  was  conducted. 

STUDY  OBJECTIVES 

o   To  compute  an  immunization  rate  based 

on  a  medical  chart  review  for  a  Houston 

area   Health   Maintenance   Organization 

insured  children  who  were  one  year  old 

in  1993. 

o     To   assess   the   validity   of   the 

immunization   rate   based   on   the   mail 

survey. 

o    To  compare  the  HMO  data  with  the 

regional  data. 

STUDY  DESIGN 

A  medical  record  review  was  conducted  on  a 
systematic  sample  of  247  children  selected  from 
the  logs  of  the  immunization  mail  survey.  The 
original  survey  was  sent  to  the  parents  of  each 
child  who  turned  one  between  July  and  October  of 
1993.  The  survey  was  sent  during  the  month  the 
child  had  his/her  one  year  birthday.  Both 
responders  to  the  mail  survey  and  non-responders 
were  included. 

To  achieve  the  second  goal  (response  bias 
analysis  and  accuracy  analysis)  another  33  who 
responded  to  the  first  study  were  selected  sys- 
tematically from  the  initial  survey  respondents. 


With  this  addition,  the  power  is  .8  to  detect  a 
15%  difference  in  immunization  rates  between  the 
respondent  group  versus  the  non- respondent 
group. 

Identification  of  Immunization  Status 

For  both  the  survey  and  the  medical  record 
validation,  a  child  is  considered  immunized  if 
he/she  had  3  DTPs,  2  OPVs,  and  3  Hibs  by  one 
year  of  age. 

In  the  medical  record  review,  these 
immunizations  are  counted  as  valid  if  the  date 
is  listed  in  the  chart.  Shots  past  the  one  year 
birthday  were  not  counted. 

Medical  Record  Review 

Each  child's  current  and  historical  Primary 
Care  Physicians  (PCPs)  were  identified  and  con- 
tacted by  letter  and  then  by  telephone.  If  there 
was  no  record  of  the  immunization,  these  doctors 
were  asked  if  there  was  a  notation  in  the  chart 
of  a  previous  doctor.  If  there  was  no  notation, 
the  parents  were  contacted  by  telephone  and  by 
letter  and  asked  where  their  child  received 
his/her  immunization.  All  identified  non-HMO 
physicians  were  contacted  by  telephone.  A  child 
was  counted  as  immunized  if  the  office  record 
noted  the  date  of  immunization. 

Regional  Comparison 

The  literature  was  scanned  to  identify 
studies  with  a  similar  definition  of  immuniza- 
tion. Local  and  state  health  departments  were 
contacted  to  identify  studies  that  may  not  have 
been  published. 

RESULTS  and  CONCLUSIONS 

The  immunization  rate  using  medical  records 
as  the  source  was  76%,  lower  than  the  survey 
percent  of  89%. 

Validation  Analysis 

To  evaluate  response  bias,  the  sample  was 
divided  into  two  groups,  respondents  to  the  mail 
survey  versus  non  respondents.  For  the 
respondents,  the  immunization  rate  was  79%  and 
for  the  non- respondents  the  rate  was  a  little 
lower  at  74%.  Using  a  Chi-square  test,  the  dif- 
ference was  not  significant  (Chi  - square  =  0 . 64 , 
p=0.42,  Table  1) . 

Discrepancies  between  the  medical  record 
review  were  analyzed.  91  medical  record  study 
members  responded  to  the  initial  survey.  Of 
these,  the  "immunization  status"  from  both 
sources  for  75  were  the  identical.  There  are  two 
types  of  discrepancies:  survey  notes  "immunized" 
while  the  medical  record  review  indicated  "not 
immunized"  or  the  reverse.  Of  the  91,  14%  (13) 
had  the  former  discrepancy  and  3%  (3)  had  the 
latter.  These  differences  were  not  due  to  chance 
according  to  the  McNemar  test,  p=0.02.  Table  2. 
This  test  is  sensitive  to  differences  in 
imbalances  in  the  types  of  discrepancies. 

C.  Hanson,  MD,  Texas  Children's  Hospital, 
found  similar  results  in  a  discrepancy  analysis 
of  a  prospective  immunization  survey  of  children 
who  were  18  to  24  months  of  age  during  January 
through  May,  1993  (4).  Of  the  parents  who 
reported  their  children  as  fully  immunized,  only 
43%  were  accurate  while  98%  of  the  parents  who 
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identified  their  children  as  lacking  immuniza- 
tions were  accurate. 

Regional  Comparison 

Most  of  the  immunization  studies  in  the 
literature  listed  results  for  two  year  old  chil- 
dren and  not  one  year  old  children.  To  find  a 
regional  comparison,  city,  county,  and  state 
health  departments  were  called.  Two  studies 
which  have  not  yet  been  published  were  identi- 
fied with  results  of  children  who  are  one  year 
of  age. 

The  Texas  Health  Department  conducted  a 
population  based  study  of  all  Texas  children 
under  age  two  in  1994  (5).  A  preliminary  anal- 
ysis found  that  70%  of  the  one  year  old  children 
were  up-to-date  by  their  first  birthday  with  3 
DTPs  and  2  PVs . 

In  a  retrospective  study  conducted  by  the 
Immunization  Division  of  the  Texas  Department  of 
Health  in  1994,  60%  (95%  C:  56%  to  64%)  of  the 
Texas  kindergarten  children  were  immunized  with 
3  DTPs  and  2  polio  vaccines  within  366  days  of 
birth  (6).  In  the  latter  study  the  birth  cohorts 
were  1988-89  while  the  HMO  study  birth  cohort 
was  1992. 

According  to  this  immunization  criteria  and 
using  the  medical  record  review  data,  the 
immunization  rate  for  the  Houston  HMO  was  81% 
(95%  CI:  77%  to  85%).  The  Houston  HMO  has  a 
statistically  significant  higher  rate  than  the 
state  as  a  whole  using  either  study  as  the 
comparison. 

The  Houston  immunization  rate  is  lower  than 
the  state  as  a  whole.  In  1993  the  Immunization 
Division  of  the  Texas  Department  of  Health  con- 
ducted a  study  similar  in  design  to  the  1994. 
For  two  year  old  children,  the  immunization  rate 
was  42%  for  the  state  as  a  whole  and  for  the 
Houston  region  the  rate  was  31%  in  1993  (7). 
This  is  the  lowest  regional  rate  in  the  state. 

The  Houston  HMO  had  a  higher  immunization 
rate  than  the  state  as  a  whole  because  HMO's 
reduce  the  effect  of  two  common  barriers.  The 
costs  are  relatively  inexpensive.  Typically,  the 
charge  is  an  office  co-pay  of  $5  to  $10.  The 
city  and  county  has  55  mobile  sites  and  12  (plus 
county)  health  centers  with  free  locations  who 
immunize  children  on  a  first  come  first  serve 
basis.  This  HMO  has  approximately  500  physician 
practitioners  and  170  pediatricians  in  the  Hous- 
ton area  who  schedule  appointments. 

QUALITY  IMPROVEMENT  ACTIVITIES 

Even  at  76%,  the  immunization  rate  for  one 
year  old  children  needs  improvement.  Three  major 
efforts  were  implemented  by  the  HMO  to  increase 
immunization  rates. 

An  article  was  written  in  the  member  maga- 
zine to  emphasize  the  ramifications  of  not 
immunizing.  A  tear  out  chart  that  can  be  updated 
and  kept  in  a  mothers  purse  or  father's  wallet 
was  inserted  in  the  magazine.  The  parent  was 
encouraged  to  have  the  physician  fill  out  the 
card. 

A  physician  letter  was  published  in  the  pro- 
vider newsletter.  This  letter  emphasized  the 
importance  of  record  keeping  which  would  allow 
the  physician  to  easily  tell  whether  the  child 
was  "up-to-date"  or  not. 

A  reminder  post  card  was  developed  and  sent 
to  the  parents  of  one  year  old  children. 

The  reasons  for  giving  a  tear  out  chart  and 
for  sending  the  reminder  postcards  are  supported 
by  research.  A  recently  published  study  of 
immunizations  in  an  HMO  has  found  that  more  than 
one  third  of  parents  did  not  know  when  the  next 


immunization  was  due  (8).  The  authors  concluded 
that  "In  managed  care  settings,  which  may  cover 
increasing  numbers  of  children,  interventions 
are  needed  to  better  inform  parents  when 
immunization  rates  are  due." 


FUTURE  STUDIES 

Two  future  studies  are  planned.  They  will 
evaluate  the  immunization  rates  for  two  year  old 
children  in  1994  and  in  1995  to  determine  the 
effects  of  the  improvement  activities. 
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Table  1. 

Response  Bias  Analysis 

Medical  Record  Review  Immunization  Status  by 

Response  to  Survey 


Nonrespondents 
Respondents 

Chi -Square 
Continuity  C 
Likelihood  Ratio 

Table  2. 
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INTRODUCTION 

The  American  Academy  of  Pediatrics  (AAP) 
recommends  that  by  18  months  of  age  all  children 
complete  a  schedule  of  four  doses  of  diphtheria- 
tetanus-pertussis  (DTP)  vaccine,  three  doses  of 
oral  polio  vaccine  (OPV) ,  and  a  single  dose  of 
measles-mumps-rubella  (MMR)  vaccine.2    A  fifth 
dose  of  DTP  and  a  fourth  dose  of  OPV  are  recom- 
mended for  children  between  the  ages  of  four  and 
six  years.   Compliance  with  immunization  period- 
icity schedules  is  important  to  ensure  an  ade- 
quate antibody  response  to  each  vaccine  and  the 
earliest  possible  protection  from  preventable 
childhood  diseases. 

Studies  have  found  that,  despite  the  net 
social  benefit  of  childhood  vaccines,  many  chil- 
dren in  the  United  States  do  not  complete  this 
series  of  immunizations  at  the  recommended  ages. 
Estimates  of  the  percentage  of  two-year-olds  cur- 
rent in  their  immunizations  range  from  lows  of  40 
to  50  percent  to  highs  of  "70  to  80  percent  with 
noted  pockets  of  lower  rates  in  inner  urban  and 
rural  areas  (Lewit  &  Mullahy,  1994;  Zell  et  al., 
1994;  Hueston  et  al . ,  1994;  Bobo  et  al .  ,  1994). 
Studies  have  also  found  that  children  in  families 
with  restrictive  household  resources,  measured  by 
available  adult  time,  more  than  one  child,  and 
low  maternal  education,  have  lower  immunization 
completion  rates  (Frank  et  al . ,  1995;  Bobo  et 
al.,  1994).   Thus,  despite  the  elimination  of 
financial  barriers  for  vaccinations,  Medicaid 
children  are  at  high  risk  of  incomplete  immuniza- 
tion in  their  preschool  years. 

Medicaid  claims  are  a  potential  low-cost 
source  of  data  for  monitoring  Medicaid  children's 
compliance  with  immunization  periodicity  sched- 
ules.  However,  they  have  limitations.   The 
claims  databases  reflect  only  immunizations 
billed  and  paid  through  the  Medicaid  program. 
Thus,  they  would  not  capture  any  immunizations 
children  received  wnen  they  were  not  enrolled  in 
Medicaid  or  immunizations  children  received  dur- 
ing periods  of  Medicaid  enrollment  that  were  paid 
through  other  public  or  private  sources.   The 
extent  to  which  Medicaid  claims  data  are  missing 
immunization  data  for  child  enrollees  has  not 
been  investigated. 

The  objective  of  this  research  is  twofold: 
(1)  to  determine  whether  Medicaid  children  are 
receiving  childhood  immunizations  at  the  recommend- 
ed ages;  and  (2)  to  evaluate  the  usefulness  of  Med- 
icaid claims  data  for  monitoring  children's  immu- 
nization status.   We  examine  Medicaid  claims  for 
DTP,  OPV,  and  MMR  vaccinations  among  children  under 
seven  years  of  age  in  four  states  during  1989. 

DATA  SOURCES 

The  Medicaid  enrollment  and  claims  database, 
known  as  the  Tape-to-Tape  files,  is  the  primary 
data  source  for  this  analysis.   In  particular,  we 
used  1989  enrollment  and  claims  data  for  Califor- 
nia, Georgia,  Michigan,  and  Tennessee.   We  took 


the  number  of  months  of  Medicaid  enrollment  in 
1989  from  the  enrollment  files  and  the  number  of 
immunizations  by  type  from  the  claims  file,  where 
immunization  type  (i.e.,  DTP,  OPV,  and  MMR)  was 
identified  from  procedure  codes. 

We  also  had  data  from  the  forms  that 
providers  filled  out  during  screening  visits  for 
the  Early  Periodic  Screening,  Diagnosis,  and 
Treatment  (EPSDT)  program.   EPSDT,  the  pediatric 
component  of  the  Medicaid  program,  is  designed  to 
provide  comprehensive,  periodic  screening  ser- 
vices, as  well  as  services  to  correct  health  prob- 
lems identified  through  the  screening  services,  to 
all  Medicaid-enrolled  children.   For  Tennessee  and 
Michigan,  providers  indicated  whether  children 
were  up-to-date  in  their  immunizations.   The  Ten- 
nessee form  had  a  single  question  referring  to  all 
childhood  immunizations  while  the  Michigan  form 
had  separate  questions  for  each  of  the  major 
childhood  immunization  types. 3 

METHODOLOGY 

To  assess  Medicaid  children's  compliance 
with  the  AAP  periodicity  schedule,  we  assigned  a 
set  of  weights  to  each  child  in  the  database. 
The  weights  reflect  the  child's  expected  number 
of  immunizations  by  type  during  the  child's 
enrollment  period  in  the  analysis  year.   To  com- 
pute the  weights,  we  determined  how  many  vaccina- 
tions of  each  type  the  child  should  have  had  dur- 
ing the  year  given  his/her  age  in  months  at  the 
end  of  the  year.   (See  Table  1.)   Then,  to 
account  for  the  fact  that  many  children  were 
enrolled  for  less  than  the  full  year,  we  multi- 
plied this  number  by  the  percentage  of  months  in 
the  year  during  which  the  child  was  enrolled. 
This  adjustment  assumes  that  the  child  is  equally 
likely  to  receive  immunizations  during  periods  of 
Medicaid  enrollment  and  periods  of  disenrollment . 


TABLE  1 

American 

Academy  of 

Pediatrics 

Immunization 

Peri 

odicity 

Schedule,  1989 

Aoe  Group 

Immunizations 

<  1  year 

3 

DTP  at 

2,  4,  &  6  months 

2 

OPV  at 

2  &  4  months 

1-2  years 

1 

MMR  at 

15  months 

1 

DTP  at 

18  months 

1 

OPV  at 

18  months 

3-6  years 

1 

DTP  at 

4-6  years 

1 

OPV  at 

4-6  years 

We  then  summed  the  weights  to  obtain  the 
total  expected  number  of  immunizations  among  the 
child  population  under  study.    This  figure  is 
the  denominator  for  our  compliance  rate.   The 
numerator  is  the  sum  of  all  immunizations  received 
by  the  population  as  reflected  in  the  number  of 
billed  immunizations  in  the  claims  data. 
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jth    Immunization  j 
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x  100 
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An  overall  compliance  rate  is  computed  by 
summing  both  the  numerator  and  the  denominator 
over  the  types  of  immunizations: 


Overall    Immunization 
Compliance   Rate 


II  i 


ij 


II  I 

3   i 


100 


ij 


Because  a  number  of  children  received  immu- 
nizations later  than  recommended  and,  therefore, 
were  not  truly  in  compliance,  we  recomputed  the 
compliance  rate  counting  only  those  immunizations 
that  fell  within  the  recommended  age  range.   We 
show  both  sets  of  rates  rather  than  just  the  age- 
appropriate  rates  because  the  extent  to  which  the 
Medicaid  program  allows  children  to  catch  up  on 
missed  immunizations  is  an  important  measure  of 
the  success  of  the  Medicaid  program  in  reaching 
children  who  otherwise  would  not  receive  these 
immunizations . 

Because  most  childhood  immunizations  should 
be  received  before  the  age  of  six  years  and 
because  most  states  require  proof  of  immunization 
for  school  entry,  we  limit  our  presentation  to 
children  six  years  of  age  and  under.   We  break 
this  age  range  out  into  three  age  groups:   (1) 
children  under  one  year  of  age  —  the  age  at  which 
the  greatest  number  of  visits  to  the  physician's 
office  are  required  for  immunizations;  (2)  chil- 
dren from  one  to  two  years  of  age  —  the  end  of 
the  target  age  range  for  the  Healthy  People  2000 
goal  of  90  percent  compliance;  and  (3)  children 
from  three  to  six  years  of  age  —  the  ages  at 
which  children  enter  school. 

Then,  to  determine  the  extent  to  which  low 
compliance  rates  are  due  to  missed  immunizations 
versus  missing  data,  we  computed  immunization 
compliance  rates  for  children  who  had  at  least 
one  EPSDT  visit  during  1989  and  compared  them  to 
the  percentages  of  children  that  providers  found 
to  be  up-to-date  in  their  childhood  immunizations 
by  their  last  EPSDT  visit  that  year. 

FINDINGS 

Table  2  shows  the  overall  compliance  rates, 
computed  with  all  DTP,  OPV,  and  MMR  vaccinations. 
Compliance  increases  with  age.   Infants  who  have 
the  greatest  number  of  recommended  immunizations 
have  the  lowest  immunization  compliance  rates 
while  children  aged  three  to  six  years  have  the 
fewest  number  of  recommended  vaccinations  and  the 
highest  compliance  rates.   In  fact,  the  compli- 
ance rates  for  the  three-to-six-year-olds  is 
greater  than  100  percent,  suggesting  that  these 
children  received  more  than  the  recommended  num- 
ber of  immunizations  for  their  age  group.   Pre- 


TABLE  2 

Immunization  Compliance  Rates 

for  1989 

Using  All 

DTP,  OPV, 

and  MMR  Vaccinations 

State 

<  I   Year 

1-2  Years 

3-6  Years 

California 

46.0% 

79.0% 

126.5% 

Georgia 

42.8 

72.3 

119.6 

Michigan 

38.0 

60.5 

105.9 

Tennessee 

49.3 

71.3 

93.6 

sumably,  these  children  are  catching  up  on  immu- 
nizations missed  in  their  earlier  years  in  order 
to  qualify  for  school  entry. 

This  hypothesis  is  confirmed  in  Table  3 
where  we  present  immunization  compliance  rates 
based  only  on  age-appropriate  immunizations.   The 
rates  for  three-to-six-year-olds  drop  25  to  40 
percentage  points  in  each  state.  The  rates  for 
one-to-two-year-olds  also  drop  substantially, 
suggesting  that  these  children  also  received 
immunizations  missed  during  their  infancy. 


TABLE  3 

Immunization  Compliance  Rates  for  1989 

Using  Age -Appropriate  DTP,  OPV, 

and  MMR  Vaccinations  Only 


State 

California 

Georgia 

Michigan 

Tennessee 


<  1  Year   1-2  Years   3-6  Years 


44.4? 
41.5 
37.2 
46.6 


56.7% 
53.0 
44.1 
52.7 


85.7? 
83.9 
67.1 
67.9 


To  determine  the  extent  to  which  Medicaid 
children  received  immunizations  under  other  pro- 
grams in  1989,  we  computed  compliance  rates  for 
the  subset  of  Medicaid  infants  and  children  aged 
one  to  two  years  with  visits  through  the  EPSDT 
program.   In  Table  4,  we  show  these  rates  and  the 
percentages  of  children  that  providers  noted  were 
up-to-date  in  their  immunizations  at  their  last 
EPSDT  visit  in  1989. 


TABLE  4 

Immunization  Compliance  Rates1 

for  EPSDT  Participants 

and  Percentage  of  Participants  Up-to-Date 

in  Immunizations 

According  to  Their  EPSDT  Screening  Form  in  1989 


State 

<  1 

1-2 

3-6 

Year 

Years 

Years 

Michigan 

Compliance  rate 

47.5% 

57.9% 

- 

Percent  up-to-date 

80.5 

75.7 

85.5 

Tennessee 

Compliance  rate 

63.8 

73.4 

- 

Percent  up-to-date 

82.5 

78.4 

80.5 

1    Only  age-appropriate  DTP,  OPV,  and  MMR 
immunizations  have  been  used  to  compute  these 
compliance  rates. 


These  data  show  that,  for  infants  with  at 
least  one  EPSDT  visit  in  Michigan  during  1989, 
nearly  48  percent  of  recommended  immunizations 
were  provided  through  the  Medicaid  program.   In 
Tennessee,  Medicaid-enrolled  infants  received 
almost  64  percent  of  recommended  immunizations 
through  the  Medicaid  program.   At  the  same  time, 
more  than  80  percent  of  infants  in  both  states 
were  reported  to  be  current  in  their  immuniza- 
tions by  their  last  EPSDT  visit  during  the  year  - 
that  is,  they  reportedly  received  all  of  their 
recommended  immunizations.   If  all  of  these  immu- 
nizations were  billed  through  Medicaid,  we  would 
expect  to  see  a  compliance  rate  over  80  percent. 
Because  the  computed  compliance  rates  are  lower, 
these  children  must  be  obtaining  their  immuniza- 
tions through  other  programs. 

The  discrepancy  between  the  percentage  of 
billed  immunizations  and  the  percentage  of  chil- 
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Iran  wh(-  had  i-pcpivpci  .ill  men  immunizat  ions  was 
smallei  tot  in  Ldren  aged  one  to  two  years.  Nev- 
artheless,  these  data  indicate  that   in  Michigan 

u   Least    »ne  ro-two-year-olds  were  also  receiv 
ing  a  substantial  number  ot  immunizations  outside 
t  i  he  Medicaid  program. 

Table  S  shows  that,  in  Michigan,  this  pat- 
tern was  consistent  across  the  three  major  types 
of  childhood  immunizations.   In  addition,  the  70 
percent  of  one-to-two-year-olds  and  the  higher  93 
percent  of  three-to-six-year-olds  current  in  the 
MMR  vaccination  confirm  that  the  older  children 
are  receiving  immunizations  missed  during  the 
earlier  years.   Recall  that  the  AAP  recommends  an 
MMR  vaccination  at  age  18  months,  but  none  from 
age  three  to  six  years  (see  Table  1) . 


TABLE  5 

Immunization  Compliance  Rates1 

for  EPSDT  Participants 

and  Percentage  of  Participants  Up-to-Date 

in  Immunizations 

According  to  Their  EPSDT  Screening  Form 

by  Immunization  Type 

In  Michigan  1989 

Type  of 

Immunization 

Diphtheria-tetanus -pertussis 

Compliance  rate 

Percent  up-to-date 
Oral  polio  vaccine 

Compliance  rate 

Percent  up-to-date 
Measles,  mumps  and  rubella 

Compliance  rate 

Percent  up-to-date 

1  Only  age-appropriate  DTP,  OPV,  and  MMR 
immunizations  have  been  used  to  compute  these 
compliance  rates. 


<  1 

1-2 

3-6 

Year 

Years 

Years 

rtussis 

45.0% 

61.5% 

- 

80.8 

79.0 

86.3 

50.3 

58.5 

_ 

82.2 

80.0 

86.9 

43.1 

_ 

- 

70.2 

92.7 

CONCLUSIONS 

Two  major  conclusions  can  be  drawn  from  this 
analysis    first   Medicaid  children  received  ^ 
substantial  number  ot  immunizations  that  were  not 
billed  through  the  Medicaid  program    Thus,  while 
Medicaid  "laimf-  date   an  dc-  used  to  snow  trends 
across  groups  ot  children  ano  pernaps  over  time, 
thev  cannot  be  used  to  determine  immunization 
levels  among  a  population  ot  children. 

Second,  a  substantial  number  of  immuniza- 
tions beyond  those  recommended  for  their  age 
group  were  received  by  Medicaid  children  aged 
three  to  six  years.   These  billings  appear  to  be 
a  result  of  children's  catching  up  on  missed 
immunizations  from  earlier  ages  so  that  they  may 
enter  school.   Thus,  the  extent  to  which  children 
aged  three  to  six  years  receive  immunizations 
beyond  those  recommended  for  their  age  group  pro- 
vides a  measure  of  the  extent  to  which  children 
missed  immunizations  at  earlier  ages.   Tracking 
this  measure  over  time  may  allow  us  to  see 
whether  we  are  getting  closer  to  the  national 
goal  of  90  percent  of  two-year-olds  fully  immu- 
nized by  the  year  1996. 
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NOTES 

1.  This  analysis  is  part  of  the  baseline  year 
findings  of  a  larger  study,  funded  by  the 
Health  Care  Financing  Administration,  that  is 
designed  to  investigate  the  impact  of  provi- 
sions in  the  1989  Omnibus  Budget  Reconcilia- 
tion Act  affecting  Medicaid  services  for  chil- 
dren . 

2.  The  AAP  also  recommends  a  Haemophilus  influen- 
za type  b  (Hib)  vaccination  at  18  months  of 
age.  However,  this  recommendation  had  just 
been  established  in  1989  and  was  not  yet  fully 
implemented  in  physicians'  practices. 

3.  California's  form  also  asked  providers  to 
check  whether  children  were  up-to-date  in 
their  childhood  immunizations  by  type  of  immu- 
nization   However   the  high  rate  of  missing 
data  in  these  variables  (greater  than  22  per- 
cent i  made  them  nard  to  interpret;  therefore, 
tney  are  not  snown  tiers 

4-  Compliance  rates  tor  three-to-six-year-old 
EPSDT  participants  were  not  computed  because 
these  rates  would  not  be  valid  for  interpreta- 
tion.  Not  all  children  in  this  age  range  are 
required  to  have  immunizations  each  year. 
Thus,  including  only  children  with  EPSDT  vis- 
its and,  therefore,  a  high  probability  of  hav- 
ing an  immunization  would  bias  the  rate 
upward.   Furthermore,  the  enrollment  duration 
adjustment  assumes  that  the  child  is  equally 
likely  to  have  an  immunization  during  periods 
of  enrollment  and  disenrollment .   This  is 
obviously  not  true  if  the  sample  is  restricted 
to  children  with  EPSDT  visits. 
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The  goal  of  the  nation  for  the  year  2000  is 
to  have  a  90%  immunization  rate  of  two  year  old 
children.  The  unchallenged  measurement  tool  for 
this  objective  is  a  survey  of  the  two  year  old 
birth  cohort.  There  may  be  a  better  way  to 
measure  this  and  other  critical  points  in 
childhood  vaccination  using  an  immunization 
registry.  However,  the  development  and  use  of  a 
registry  itself  creates  new  problems  which  could 
limit  the  usefulness  of  a  registry. 

Most  immunization  programs  rely  heavily  on 
surveys  to  provide  them  and  the  immunization 
workers  in  the  field  with  the  vital  data  they 
need  to  monitor  and  improve  immunization  rates. 
In  Mississippi,  all  health  department  clinics 
utilize  the  same  state  provided  data.  Apart  from 
a  single  report  each  month  of  new  births,  all  of 
the  remaining  data  for  program  management  comes 
from  surveys  and  reports  completed  by  parties 
interested  in  the  effort. 

Current  Immunisation  Information  System 

Birth  printout  from  vital  statistics 
(sanitized) .  This  provides  a  snapshot  of  the 
population  at  that  time,  which  gets  less  accurate 
as  the  cohort  ages.  The  printout  is  not 
complete,  as  there  are  always  delayed 
registrations.  This  is  manual  and  does  not  feed 
an  automated  system  that  will  reduce  time  and 
errors  identifying  children  who  miss 
vaccinations. 

Yellow  shot  card  for  each  health  department 
patient  who  receives  childhood  vaccines.  In 
Mississippi  clinics,  these  cards  are  filed  by 
date  to  serve  as  a  tickler  for  follow-ups  and 
reminders.  These  cards  are  manually  maintained 
and  are  subject  to  loss  and  misfiling  in  addition 
to  data  capture  errors.  These  cards  are  only 
completed  in  the  health  departments  for  clinic 
patients,  and  not  available  to  the  staff  for 
children  of  private  doctors  and  clinics.  The 
cards  are  also  not  readily  available  across 
clinics.   Thus  the  need  for  a  central  registry. 

Vaccine       Administration       Record.  This 

federally  mandated  record  is  completed  by  the 
nurse  for  each  vaccine  type  showing  batch/lot, 
site  of  administration,  manufacturer  ,  as  well  as 
warnings.  This  form  captures  data  available  in 
the  registry  and  is  a  potential  for  elimination. 

Yellow  card  maintained  by  parents.  This 
card  is  completed  by  the  nurse  each  time  a  shot 
is  given  to  the  child,  whether  in  a  health 
department  clinic  or  in  a  private  office.  This 
card  is  subject  to  loss  and  more  frequently  to 
being  forgotten  at  the  visit.  The  information 
may  not  be  current  and  is  definitely  not  readily 
available  to  immunization  workers  to  determine 
status. 

Blue  and  other  colored  compliance  forms. 
These  forms  are  completed  by  providers  and  serve 
as  formal  notification  to  institutions  that  the 
child  is  compliant.  They  contain  minimal  data 
and  are  either  retained  by  the  institution  or 
lost  by  the  parents. 

Two  year  old  survey.  This  survey  is  of  the 
birth  cohort  at  24  months  of  age  and  is  the  basic 
measurement  tool  for  programs  across  the  country. 
In  Mississippi,  this  survey  is  stratified  across 
the  nine  public  health  districts  and  then 
randomly  sampled  based  on  population  and 
immunization  levels.  Districts  with  low 
populations  and  poor  immunization  levels  have  the 
highest  sampling  rate  of  7.5%.  The  lowest  rate 
is  1.9%  for  a  district  with  high  population  and 
good  immunization  level.  Statewide  1,343 
children  were  sampled  out  of  a  cohort  of  41,000. 


The  limitations  of  this  survey  are  errors  in  data 
collection,  incomplete  data  due  to  poor 
documentation  and  delays  due  to  the  nature  of  the 
survey.  The  majority  of  the  problems  with 
immunization  rates  in  Mississippi  occur  between 
the  first  and  second  birthdays.  Since  all 
immunizations  should  be  completed  by  the  15th 
month,  and  the  second  survey  is  done  up  to  35 
months  of  age.  There  is  a  20  month  lag  between 
scheduled  immunization  and  review.  Because  it  is 
a  sample,  it  is  not  useful  for  tracking  of 
individuals  but  only  to  provide  directions  for 
changes  in  the  system. 

School  Compliance  Reports  -  Preliminary  and 
Final.  All  schools  in  the  state  complete  a 
report  manually  which  details  by  school,  grade 
and  certificate  type,  the  status  of  students. 
This  report  provides  to  the  immunization  program 
information  that  is  already  known  to  the  health 
department  clinics  for  at  least  50%  of  students. 
The  agency  has  no  systems  is  place  that  can 
retrieve  these  data.  Similarly,  neither  the 
schools  nor  the  state  education  department  have 
systems  to  generate  these  data. 

First  Grade  School  Validation  Survey. 
Based  on  the  school  compliance  reports,  a 
stratified  random  sample  of  schools  and  first 
grade  classes  is  drawn  and  students  sampled 
within  these  classes.  The  program  workers  then 
determine  the  status  of  the  children  in  the 
sample.  At  the  same  time,  the  status  of  these 
students  at  2  years  of  age  is  also  determined 
retrospectively.  This  sample  frame  generated 
5,500  students  last  year.  This  sample  also  helps 
measure  the  rates  at  the  two  check  points  and 
determine  system  points  of  failure. 

College  Immunization  Survey.  This  is  a 
random  sample  of  first  time  admissions  to 
colleges  in  the  state  for  mumps  and  rubella 
compliance.  The  sample  of  20%  in  1994  generated 
4,345  student  records  that  were  followed  back  by 
immunization  workers. 

Child  Care  Validation  Survey.  This  survey 
is  a  random  sample  of  10%  of  the  child  care 
facilities  licensed  by  the  state.  Within  each 
facility,  20%  of  the  children  over  age  two  were 
chosen  for  a  total  sample  of  1,143.  This  survey 
obviously  measures  the  levels  in  the  day  care 
centers  that  are  prime  targets  for  disease 
transmission. 

Statewide  Immunization  Registry 

The  obvious  solution  for  a  centralized 
state  like  Mississippi  is  to  develop  a  statewide 
immunization  registry.  Several  factors  converged 
in  1995  to  enable  us  to  get  into  a  statewide 
registry.  The  agency  had  been  working  for 
several  years  to  automate  its  clinic  functions, 
one  of  which  was  immunization.  The  basic  system 
was  completed  and  installed  statewide  by  early 
1995.  At  the  same  time,  the  state  legislature 
mandated  a  statewide  immunization  registry 
effective  January  1995.  This  law  required  all 
doctors  giving  immunizations  to  report  data  to 
the  state  registry  and  to  be  provided  current 
status  from  the  registry.  The  internal  clinic 
system  (PIMS)  is  integrated  for  all  health 
department  programs,  including  immunization, 
which  meant  that  it  would  have  to  be  modified  to 
protect  non-immunization  data  from  external 
users.  This  could  not  be  done  immediately,  so 
the  immunization  program  has  provided  for  phone 
operators  and  mail-in  reporting  of  immunizations 
until  the  system  has  been  converted.  The  state 
is  in  a  good  technical  position  to  move  into  a 
statewide  registry  because  it  already  provides 
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over  75%  of  all  immunizations  given,  which  with 
our  rate  of  73%,  amounts  to  55%  of  the  total 
population. 

The  software  already  provides  for  the 
agency  the  basic  features  of  a  registry  such  as 
data  collection,  reminders,  internal  logic  to 
determine  validity  of  doses,  doses  administered 
reporting  and  VFC  reporting.  Before  external 
providers  could  utilize  the  system,  firewall 
software  had  to  be  written  to  protect  internal 
clinic  data  from  the  external  users  of 
immunization  only  data.  This  project  is  being 
funded  by  CDC  through  the  immunization  program 
and  has  been  stalled  for  several  months.  Once 
this  has  been  done,  the  real  problem  begins  with 
training  the  several  hundred  doctors'  offices  who 
give  shots  how  to  use  the  system. 

The  registry  has  the  potential  to 
drastically  change  the  way  immunization  status  is 
determined  as  well  as  improving  the  efficiency  of 
the  immunization  program  workers. 

The  system  will  eliminate  the  need  for  blue 
slips  when  the  schools  can  access  the  system. 
The  system  already  has  automated  blue  slip 
production.  It  also  can  potentially  eliminate 
the  need  for  birth  certificate  hard  copies  for 
school  entry.  The  major  barrier  to  implementing 
this  technology  is  the  reduction  in  revenue  to 
the  vital  statistics  program.  The  system  will 
produce  call  lists,  record  pull  lists  and  any 
other  report  which  can  be  logically  defined  to 
the  system.  This  will  give  the  program  staff  the 
ability  to  focus  the  attention  on  any  group  of 
the  population  which  they  feel  can  improve. 

Automation  of  the  system  has  the  potential 
to  reduce  the  paperwork  in  the  clinics.  The 
yellow  tickler  cards  and  the  vaccine 
administration  record  are  candidates  for 
elimination.  The  major  problem  at  this  stage  is 
procedural,  the  current  form  requires  the  nurse's 
signature.  The  rest  of  the  data  on  the  form  is 
being  captured  in  the  registry.  The  benefits  to 
the  parents  in  not  having  to  worry  about  the 
whereabouts  of  their  yellow  card  are  many. 
Imagine  being  able  to  have  any  doctor  or  nurse, 
school  or  daycare  pull  up  the  status  of  my  child 
from  any  PC. 

The  surveys  will  change  dramatically 
because  the  population  is  basically  in  the 
system,  at  least  for  the  two  year  olds  and  the 
six  year  olds.  The  registry  has  all  of  the 
health  department  records  in  the  system  from  1992 
forward.  Later  this  year  we  will  receive  from 
Social  Security  Administration  (SSA)  a  list  of 
all  records  from  1990  forward.  This  will  also 
help  populate  the  registry.  The  immunization 
field  workers  will  continue  to  review  records  for 
completeness,  but  the  objective  will  be  redefined 
from  measurement  to  one  of  data  quality  and 
completeness. 

Problems  and  Opportunities  of  a   Statewide 
Registry 

Accuracy  and  Completeness.  The  registry 
will  be  built  from  the  database  for  newborns,  the 
vital  statistics  birth  file.  This  file  in 
Mississippi  is  mostly  created  electronically  via 
the  EBC  process,  in  itself  a  more  accurate  way  of 
obtaining  data.  These  data  are  then  sent  to  SSA 
where  the  Social  Security  Number  (SSN)  is  created 
and  added  to  the  records  of  those  children  whose 
parents  request  electronic  generation  of  SSNs. 
This  file  when  received  back  in  the  state, 
contains  the  accurate  name,  date  of  birth  and 
SSN.  The  population  of  the  cohort  is  therefore 
built  from  accurate  data  with  no  possibility  of 
error.  Errors  can  only  come  from  changes  in 
items  which  are  not  reflected  in  the  file  or  from 
incorrect  immunization  events  recording  in  the 
file.  Of  course,  migration  over  time  will  also 
be  a  possible  source  of  errors. 

The  other  issue  for  Mississippi  is  the  high 
percentage  of  immunizations  given  at  the  health 
department  clinics.  Currently,  about  two  thirds 
of  all  immunizations  are  given  in  these 
locations.   All  of  those  events  are  recorded  in 


the  PIMS  based  registry.  This  means  that  as  long 
as  internal  data  quality  is  high,  the  program 
field  workers  can  concentrate  on  improving  the 
quality  of  the  30  %  not  internal  to  the  health 
department . 

Single  Database .  The  system  is  based  on  a 
mainframe  which  contains  the  one  file  for  the 
only  registry  for  the  state.  There  are  no 
separate  files  to  keep  in  synch  and  no  multiple 
queries  to  make  to  completely  identify  or  exclude 
a  child.  This  makes  the  system  simple  to  develop 
around.  There  is  only  a  computer  system  to 
relate  to  and  only  one  file  to  look  up.  Training 
and  setup  for  retrieval  is  simple,  although  the 
mainframe  has  several  options  for  access. 

Mainframe  Based.  The  mainframe  has  some 
advantages.  It  has  several  different  options 
available  for  access  from  almost  any  type  of 
computer  system.  The  technology  is  readily 
available  for  linkages  from  mainframes, 
minicomputers,  server  based  LANs  and  single  PCs. 
The  state  mainframe  supports  access  from  PCs  via 
asynchronous  dialup  and  via  TCP/IP  Internet  type 
connection.  The  system  is  also  online  to  the 
Internet  and  can  be  addressed  this  way.  The 
mainframe  is  also  a  reasonably  secure  system, 
particularly  for  the  medium  level  of  security 
data  in  a  registry.  This  system  is  supported 
already  and  the  agency  has  minimum  work  to  do  in 
order  to  maintain  the  system  on  the  mainframe. 

System  Interfaces .  There  are  many  larger 
private  operations  that  are  automated  and  would 
prefer  to  have  automated  interfaces  to  the 
registry.  This  is  technologically  possible,  but 
will  take  a  concerted  effort  on  the  part  of  the 
immunization  program  or  the  private  clinics.  The 
software  developer  of  the  registry  has  a  common 
interface  protocol  defined  that  will  make  it 
easier  for  this  type  of  high  level  interface  to 
be  implemented. 

Private     Physician     Training.  There  are 

about  400  private  physicians  in  Mississippi  who 
routinely  give  immunizations.  The  training 
effort  for  the  staff  in  these  offices  is 
substantial,  both  initially  and  ongoing.  While 
the  software  is  not  complicated  or  difficult  to 
learn,  there  is  still  a  learning  curve  for  the 
office  and  support  staff  in  the  physicians 
offices.  We  would  be  pleased  if  all  offices 
could  be  trained  initially  within  two  years.  At 
this  point,  the  staffing  for  this  effort  has  not 
been  acquired. 

Education  Department  Linkage.  Exploratory 
meetings  have  been  held  with  the  State  Department 
of  Education  who  would  like  to  develop  an 
electronic  student  record  on  a  statewide  basis. 
This  record  would  be  maintained  by  the  State 
department  and  accessed  by  all  schools  in  the 
state.  When  students  transfer,  the  electronic 
record  would  still  be  available.  A  portion  of 
this  record  contains  immunization  and  birth 
certificate  data.  Obviously,  these  data  should 
come  from  the  registry,  and  could  either  be 
transferred  to  the  education  file  or  simply  be  a 
link  between  the  two  systems.  Within  the  next 
two  years,  all  of  the  schools  in  the  state  will 
be  online  to  the  education  system  via  an  Internet 
type  network.  During  that  same  time  frame,  the 
registry  should  be  complete  for  over  90%  of  the 
birth  cohort,  and  7  5%  complete  for  the  5  year  old 
cohort.  It  is  possible  that  K5  children  will  be 
checked  for  date  of  birth  and  Immunization 
compliance  without  having  to  access  a  health 
department  system  directly. 

Junior  and  Senior  College  Systems.  When 
this  concept  was  presented  to  a  group  of  college 
presidents,  they  wanted  access  to  the  system 
right  away.  They  are  not  willing  to  wait  for  15 
years  for  the  registry  to  age  gracefully  so  their 
age  group  will  be  represented.  This  offers  us  an 
opportunity  to  see  if  the  colleges  can  find  a  way 
to  enter  their  freshmen's  data  as  their  staff  are 
determining  compliance.  We  will  be  looking  for 
funding  to  provide  to  colleges  for  historical 
data  entry.  We  have  over  half  of  the  data,  which 
we  can  get  in  easily  with  some  data  entry  help. 
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Students  from  colleges  may  be  better  used  to 
track  down  the  records  of  the  private  providers. 

Automation   of  Private      Offices.  This 

process  will  be  tedious  and  potentially  very 
expensive,  based  on  the  cost  per  child  immunized. 
Other  registries  have  found  that  providing 
software  that  will  do  more  than  just  access  the 
registry  can  increase  physician  involvement.  The 
immunization  program  will  be  working  to  identify 
software  providers  who  will  sell  us  a  statewide 
license  with  an  interface  to  our  registry.  The 
vendor  will  also  install  and  train  worker  in  the 
use  of  the  system. 

Day  Care  Facilities.  The  12  to  24  month 
old  children  are  not  in  school  during  this  time, 
but  either  at  home  or  in  day  care  facilities. 
While  the  state  department  of  health  licenses 
these  facilities,  there  is  no  automated  record  of 
the  children  in  these  facilities.  Most  of  the 
facilities  are  not  automated,  nor  are  they 
expected  to  be  in  the  near  future.  We  anticipate 
that  our  inspectors,  both  day  care  and 
immunization,  will  be  equipped  with  laptops  that 
will  be  network  enabled  and  could  verify  the 
status  of  day  care  children  during  their 
inspections.  The  larger  facilities  may  be 
sampled,  but  the  smaller  ones  could  be  100% 
checked  for  compliance  using  the  registry. 

Data  Quality  Control.  Followup  reviews  of 
historical  immunization  data  entry  in  the  health 
departments  have  shown  large  percentages  of  data 
entry  errors.  Some  of  these  are  unavoidable 
while  others  are  definitely  due  to  carelessness. 
The  degree  of  errors  has  made  us  rethink  how  we 
will  ensure  the  quality  of  the  data  being 
entered.  For  our  own  employees  (all  staff  in  the 
health  department  clinics  are  state  agency 
employees)  we  will  incorporate  data  quality 
standards  into  performance  appraisals.  We  will 
also  require  nurses  to  review  and  check  data 
entry,  particularly  the  historical  data.  Based 
on  these  data,  we  will  look  toward  contractors 
for  the  bulk  of  future  historical  data  entry. 
SUMMARY 

The  immunization  program  samples  annually 
with  labor  intensive  follow-back  over  12,000 
records  each  year.  The  state  immunization  rate 
exceeds  70%  on  a  base  of  approximately  40,000 
births.  If  the  existing  workers  were  to  focus 
their  follow-back  efforts  on  the  missing  30%, 
this  would  approximate  the  12,000  that  are 
currently  being  sampled.  This  is  ideal,  but  the 
reality  is  that  the  field  staff  will  have  to 
continue  to  sample  the  population  file  to  ensure 
that  the  quality  of  the  data  in  the  registry  file 
is  of  acceptable  quality.  The  staff  will  shift 
the  focus  of  the  followback  effort  from 
measurement  to  quality  control. 

The  effort  required  to  bring  the  majority 
of  the  private  providers  is  significant,  but 
necessary  to  make  the  registry  fully  productive. 
The  state  will  have  to  experiment  with  various 
incentives  which  will  encourage  the  participation 
of  the  private  sector.  Historical  data  entry  has 
been  a  major  impediment,  but  innovative  funding 
and  contract  sources  may  speed  up  this  vital 
phase  of  the  registry's  development. 
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NON-FATAL   FIREARM   INJURIES 


NEW   ZEALAND   COMPARED   WITH   USA. 


John  D  Langley,  University  of  Otago 

Joseph  L  Annest 

Stephen  W  Marshall 

Robyn  N  Norton 


Aim:  To  compare  the  epidemiology  of  serious 
non-fatal  firearm  injury  in  New  Zealand  (NZ)  to 
that  in  United  States  of  America  (USA) . 
Method:  New  Zealand  cases  were  selected  from 
the  New  Zealand  Health  Information  Service's 
hospital  inpatient  data  files  for  the  period 
1979  to  1992  inclusive.  USA  inpatient  cases 
were  selected  from  the  USA  National  Centre  for 
Injury  Prevention  and  Control  (NCIPC)  Firearm 
Injury  Surveillance  Study. 

Results:  At  22.0  per  100,000  population,  the 
USA  has  an  inpatient  injury  rate  for  nonfatal 
firearm  injuries  8  times  that  of  NZ .  In  the 
NCIPC  inpatient  series  35%  of  the  incidents 
involved  handguns  whereas  in  the  NZ  series  less 
than  2%  involved  handguns. 

In  the  NZ  series  64%  of  the  injuries  were 
considered  to  be  due  to  unintentional  events 
whereas  they  accounted  for  only  13%  in  the 
NCIPC  series  with  assault  being  the  main 
contributor  at  61%. 
Conclusions 

The  differences  between  USA  and  NZ  in  inpatient 
rates  for  overall  nonfatal  firearm  injuries  and 
for  those  associated  with  assaults  may  be 
related  to  firearm  policies  and  socio-cultural 
differences.  This  needs  to  be  investigated 
further  through  epidemiologic  studies  and 
international  comparisons. 

Introduction 

Mortality  and  morbidity  due  to  firearms  are  the 
subject  of  considerable  controversy,  especially 
in  USA.  Much  of  the  debate  has  focused  on  the 
high  and  unnecessary  number  of  fatalities  due 
to  firearms.  At  14.9  per  100,000  population  USA 
has  one  of  the  highest  firearm  mortality  rates 
in  the  world  (Kochanek  &  Hudson  1994)  . 

Until  recently  national  estimates  on  non- 
fatal firearm  injuries  have  not  been  available. 
The  aim  of  this  paper  is  to  compare  key 
findings  from  recent  NZ  and  USA  population 
based  studies  of  non-fatal  firearm  injury.  This 
comparison  was  stimulated  by  the  International 
Collaborative  Effort  on  Injury  Statistics  (NCHS 
1995)  . 

Method 

For  the  purposes  of  this  study  serious  non- 
fatal injury  was  operationally  defined  as  any 
injury  which  did  not  result  in  death  but  which 
required  inpatient  treatment  in  a  public 
hospital . 

A  firearm  related  injury  was  defined  as 
any  injury  which  resulted  from  the  discharge  of 
a  firearm.  Injuries  such  as  those  due  to  the 
direct  handling  of  cartridges,  for  example, 
were  not  included.  Similarly,  injuries  due  to 
airguns,  stun  guns,  flare  guns,  nail  guns,  and 
other  such  devices  were  also  excluded. 

The  method  for  selecting  cases  for  the  NZ 
series  is  described  in  detail  by  Langley  et  al 


(1995)  .  Very  briefly,  cases  were  selected  from 
the  New  Zealand  Health  Information  Service's 

(NZHIS)  hospital  morbidity  data  files  for  the 
period  1979  to  1992  inclusive.  All  discharges 
with  the  following  E-codes  were  selected:  E922 
Accident  caused  by  firearm  missile;  E955 
Suicide  and  self  inflicted  injury  by  firearms 
and  explosives;  E965  Assault  by  firearms  and 
explosives;  E985  Injury  by  firearms  and 
explosives,  undetermined  whether  accidentally 
or  purposely  inflicted;  and  E970  Injury  due  to 
legal  intervention  by  firearms.  Free-text 
descriptions  associated  with  each  case  were 
reviewed  in  order  to  code  the  type  of  firearm 
and  eliminate  misclassif ications .  Readmissions 
to  hospital  for  the  treatment  of  the  same 
injury  were  excluded. 

Estimates  for  the  USA  inpatient  series 
were  obtained  from  a  series  based  on  emergency 
department  visits  (NCIPC  Firearm  Injury 
Surveillance  Study) .  The  method  for  this  study 
has  been  described  in  detail  by  Annest  et.al. 

(1995)  .  Very  briefly,  data  were  obtained  from 
medical  records  for  all  firearm-related  injury 
cases  identified  using  the  National  Electronic 
Injury  Surveillance  System  (NEISS)  during  the 
study  period  of  June  1,  1992-May  31,  1993. 
NEISS   data  is  collected  at  91  hospitals  and 
provides  a  stratified,  representative 
probability  sample  of  all  attendances  at  USA 
hospitals  with  emergency  departments.  Inpatient 
cases  were  identified  from  this  series  and 
national  estimates  calculated. 

Result  s 

Between  1979  and  1992  there  were  1,239  firearm 
related  injuries  in  NZ ,  an  average  of  89  per 
year.  The  crude  morbidity  rate  for  the  study 
period  was  2.7  injuries  per  100,000  population 
(95%  CI:  2.6-2.9).  The  equivalent  estimate  for 
USA  was  22.0  per  100,000  population  (95% 
CI : 13 . 0-31 . 0)  (Table  1).  In  both  countries  male 
victims  predominated. 

The  distribution  of  incidents  shows  that 
relative  to  USA  a  disproportionate  part  of  the 
burden  in  NZ  falls  on  the  younger  age  groups. 
In  both  countries  those  in  the  15-24  year  age 
group  have  the  highest  rates  of  injury. 

There  were  marked  differences  in  the 
circumstances  of  injury.  In  the  NZ  series  64% 
of  the  injuries  were  considered  to  be  due  to 
unintentional  events  whereas  they  accounted  for 
only  13%  in  the  USA  series  with  assault  being 
the  main  contributor  at  61%. 

Similar  marked  differences  were  apparent 
with  respect  to  type  of  firearm.  In  the  USA 
inpatient  series  3  5%  of  the  incidents  involved 
handguns,  whereas  in  the  NZ  series,  less  than 
2%  involved  handguns.  Long  guns  (shotguns,  and 
rifles)  accounted  for  45%  of  all  NZ  incidents. 
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Table  1  -  Non  Fatal  Firearm  Injuries  NZ/USA 


%   Distribution 
NZ  USA 


Crude  Rates* 
NZ  USA 


(1979-92)   (1992-93) 

(1979-92)   (1992-93) 

Gender 

Males 

90.6        87.7 

5.0        39.6 

Females 

9.4        12.3 

0.5         5.3 

Age  Group 

0-14 

15-24 

25-44 

45-64 

>65 


9 

.3 

45 

.9 

35 

.2 

7 

.8 

1 

.  8 

64 

.4 

13 

.8 

15 

.4 

4.1 

40.9 

46.0 

6.9 

1.7 


1.0 
7.0 
3.3 
1.2 
0.5 


4 
63 
31 


Event 

Unintentional 

Self-inflicted 

Assault 

Legal  intervent 

Undetermined 


6.1 


13.4 
7.4 

61.1 
1.2 

16.9 


1.7 
0.4 
0.4 

0.2 


2 
1 
13 
0 
3 


Firearm 

Shotgun 
Rifle 
Handgun 
Unspecified 


25.1 

20.3 

1.6 

53.0 


7.0 

3.7 

35.2 

54.2 


0.7 
0.5 


1.4 


1 

0 

7 

11 


.5 


Total 


2.7 


22. 


*  per  100,000  pop ' n 

Discussion 

The  USA  population  rate  was  eight  times  that 
reported  for  NZ .  One  potential  explanation  for 
the  difference  is  NZ ' s  stricter  laws  regarding 
to  the  sales  and  possession  of  firearms  and  in 
particular  handguns.  Past  and  current  law  has 
restricted  handgun  ownership  to  bona- fide 
members  of  pistol  clubs. 

It  is  not  immediately  apparent  why  a 
relatively  high  proportion  of  young  New 
Zealanders  are  involved  in  firearm  incidents. 
The  reason  may  related  to  New  Zealand's 
relatively  high  youth  an  young  adult 
participation  levels  in  sporting  activities  and 
strong  tradition  in  outdoor  recreation 
activities  which  involve  firearms  (e.g.  deer 
culling,  pig  shooting)  (Reeder  et  al  1991). 

The  most  dramatic  difference  between  the 
two  countries  was  in  the  inpatient  injury  rates 
for  firearm  assault  (Table  1) ;  the  crude  rate 
for  USA  was  34  times  greater  than  that  for  NZ . 
One  possible  explanation  may  be  that  USA  has 
higher  overall  levels  of  assault  than  Zealand. 
The  absence  of  national  USA  inpatient  data 
precludes  a  comparison  of  rates  of  serious 
assault  between  the  two  countries.  USA  had  a 
1992  homicide  (all  causes)  rate  of  10.0  per 
100,000  population  (Kochanek  &  Hudson  1994) 
whereas  that  for  NZ  was  2.5  (New  Zealand  Health 
Information  Service  1994).  While  significant, 
this  suggests  that  the  difference  in  firearm 
assaults  between  the  two  countries  cannot  be 
entirely  explained  as  a  reflection  of  the 
differences  in  overall  levels  of  assault. 

The  absence  of  detail  on  the  specific 
type  of  firearm  in  over  50%  of  the  cases  in 
both  series  limits  our  ability  to  delineate  the 
reasons  for  the  differences  between  the 
countries.  For  example,  given  New  Zealand's 
stricter  handgun  laws,  victims  unintentionally 
injured  may  suppress  the  fact  that  a  handgun 
was  involved.  It  is  also  of  concern  for  both 


countries  in  that  various  strategies  are  in 
place  or  have  been  suggested  for  the  control  of 
specific  types  of  firearms. 

Most  international  comparisons  of  health 
statistics  focus  on  mortality.  This  is  because 
of  the  gravity  of  these  events,  relative 
reliability  of  the  fatality  statistics,  and  the 
absence  of  population  based  data  on  non-fatal 
outcomes  in  most  countries.  In  that  it  utilises 
morbidity  data  in  comparing  two  countries  this 
study  represents  a  significant  advancement. 
Explanations  for  the  marked  differences  in  the 
circumstances  of  firearm  injury  probably  lie 
in  a  combination  of  differences  in  firearm 
policy  and  more  general  socio-cultural  factors. 
This  needs  to  be  investigated  further  through 
more  detailed  epidemiologic  studies  and 
international  comparisons  which  include 
mortality,  morbidity  and  risk  factor  data. 
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ABSTRACT.  One  of  the  more-often 
used  measures  of  multiple 
injuries  is  the  Injury  Severity 
Score  (ISS)  which  is  the  sum  of 
squares  of  the  highest 
abbreviated  Injury  Scale  values 
in  each  of  the  three  most 
severely  injured  body  regions. 
The  ISS  is  neither  a  summary  of 
injuries  in  all  body  regions  nor 
a  comprehensive  summary  of 
injuries  in  one  body  region. 
This  paper  suggests  a  geometric 
extension  of  the  ISS  to  assess 
the  overall  severity  of  all 
injuries,  the  complete  ISS 
(CISS) .  This  measure  is  used  to 
examine  data  on  761  patients 
injured  during  a  war .  CISS  is 
used  to  predict  LOS  for  patients 
with  4  or  more  injuries.  CISS  is 
compared  to  ISS  in  order  to 
evaluate  the  importance  of  the 
new  methodology  in  patients  with 
four  or  more  injuries. Contrary 
x.o   the  ISS,  the  CISS  allows  the 
consideration  of  several 
injuries  in  the  same  body  region 
and  the  summing  of  an  unlimited 
number  of  injuries  in  all  body 
regions . 


INTRODUCTION 

The  assessment  of  the  overall 
severity  of  multiple  injuries  is 
fundamental  for  appropriate 
treatment  and  assessment 
ofmedical  care.  One  of  the  most 
important  and  more-often  used 
measures  of  multiple  injuries  is 
the  Injury  Severity  Score  (ISS) , 
first  proposed  by  Baker (1,2) . 
Determination  of  the  ISS  is 
based  on  the  Abbreviated  Injury 
Scale  (AIS) (1-9)  which  is  a 
numerical  scale  of  injury 
severity  ranging  from  1  (minor 
injury)  to  6  (maximal  injury- 
unsurvivable)  (Table  1) . 
Injuries  are  determined  for  each 
of  the  body  regions  (Table  2) . 
The  ISS  is  the  sum  of  squares  of 
the  highest  AIS  values  in  each 
of  the  three  most  severely 


injured  body  regions  (Table  3) . 
Thus,  it  summarizes  the  anatomic 
trauma  to  one  or  more  body 
regions  and  can  provide  a  scalar 
description  of  the  overall 
severity  of  injury  in  patients 
with  multiple  trauma. 

While  aimed  at  the  overall 
description  of  severity,  the  ISS 
is  not  a  comprehensive  summary 
of  injuries  in  all  body  regions 
and  may  underestimate  the  impact 
of  all  injuries  to  a  person.  The 
ISS  describes  the  anatomical 
damage  in  the  three  most 
severely  injured  body  regions 
but  its  shortcomings  are  well 
recognized  (1,10,11).  The 
inclusion  of  an  AIS  in  the  ISS 
calculations  is  relative  to  the 
AIS  in  other  regions.  The  ISS 
can  be  an  underestimate  of  the 
description  of  multiple  severe 
injuries  when  four  or  more 
injuries  occur,  because  only 
three  of  the  injuries  are 
considered. 

In  addition,  the  ISS  is  not  a 
comprehensive  summary  of 
injuries  in  one  body  region 
(1,11).  The  ISS  takes  into 
account  the  most  severe  injury 
in  three  different  body  regions. 
However,  when  two  or  more 
injuries  occur  in  one  body 
region,  the  equally  serious  or 
less  serious  injury  is  not 
considered,  although  multiple 
serious  injuries  in  the  same 
body  region  are  possible, 
particularly  in  penetrating 
abdominal  or  thoracic  injuries. 
Examination  of  causes  of 
mortality  in  3000  casualties 
with  penetrating  abdominal 
injuries  in  World  War  II 
indicated  that  "case  fatality 
rate  increased  about  15  percent 
as  each  additional  viscus  was 
injured"  (12) .  In  a  more  recent 
study,  Bellamy  and  Vayer 
concluded  that  "assigning  the 
same  risk  to  a  casualty  with 
multiple  organ  injuries  within 
one  body  region  as  to  a  casualty 
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with  a  single  organ  injury  is 
manifestly  unreasonable" (13) . 

This  paper  suggests  a  new 
approach  to  assessing  the 
overall  severity  of  injury  and 
explores  ways  to  describe  fully 
four  or  more  injuries.  CISS  is 
used  to  predict  LOS  for  patients 
with  4  or  more  injuries.  The 
findings  are  compared  to  the  ISS 
in  order  to  evaluate  the 
importance  of  the  new 
methodology . 

Geometric  analogy. 

The  ISS  is  calculated  as  the  sum 
of  squares  of  the  AIS  in  each  of 
the  three  most  severely  injured 
body  regions . 


1.  ISS  =  AIS]/ 


AIS' 


+  AIS^ 


An  equation  similar  to  the 
Pythagorean  triangle  theorem 
will  be  used  to  enable  all 
injuries  to  be  considered. 
Conceptually  the  sum  of  the 
squares  of  all  AISs  along  the 
right  angle  sides  are  considered 
to  equal  the  square  of  the 
hypotenuse  (r^)  which  is 

considered  to  be  the  magnitude 
of  the  severity  (Figure  1) : 


r2 

r3 


2  - 


=  AISi 


AIS2' 


=  r22  +AIS32   =  ISS 


Conceptually,  one  could  continue 
analogous  calculations  to 
include  additional  AISi  for  each 
injury  (i)  in  all  body  organs 
that  are  affected,  without  being 
limited  to  only  three  injuries 
in  different  organs. 

Similar  to  the  ISS,  a  complete 
ISS  (CISS)  for  i  injuries  can 
now  be  defined: 


^SSi  =r±2  =  £AISt2 


The  length  of  final  vector   is, 
in  fact,  an  overall  description 
of  the  magnitude  of  severity, 
taking  into  account  all  injuries 
to  a  person. 


EXAMPLE:  The  medical  records  of 
all  patients  admitted  to  the 
Emergency  Department  of  Rambam 
Medical  Center  (RMC)  during  the 
war  in  Lebanon  from  June  6-30, 


1982,  were  reviewed. All  data, 
including  medical 
characteristics,  were  recorded, 
using  a  detailed  coded 
questionnaire (14) .All  780 
casualties  were  included  in  this 
study.  Five  patients  who  had  an 
AIS  of  6  were  excluded  from  the 
analyses  because  the  comparison 
of  their  ISS  (assigned  to  be  75 
per  definition)  and  the  CISS  was 
not  meaningful.  For  14  patients 
who  were  hospitalized  for 
reasons  other  than  trauma,  an 
ISS  could  not  be  determined  and 
they  were  also  excluded  from  the 
analyses,  leaving  a  total  of  761 
patients  to  be  considered.  Four 
physicians  independently 
examined  each  medical  record  and 
coded  questionnaire  to  determine 
the  ISS .  This  team  was  not  aware 
of  the  planned  study  to  compare 
ISS  and  CISS.  The  1985  AIS 
revision  was  used  to  determine 
the  AIS  score  and  then  to 
calculate  the  ISS(7,  14).  In 
cases  of  disagreement,  the  ISS 
value  was  determined  by 
consensus . 

STATISTICAL  ANALYSES 

The  analyses  explored  the 
differences  between  ISS  and  CISS 
for  individuals  with  different 
numbers  of  injuries.  The  ISS  is 
a  measure  on  an  ordinal  scale  so 
that  appropriate  statistics  for 
ordinal  measures  should  be  used 
in  statistical  analyses  (1,  10, 
16) .  Thus,  the  medians  are 
examined  here  to  get  more 
insight  into  the  CISS 
methodology . 

The  statistics  were  calculated 
using  SPSS  TABLES,  and 
Wilcoxon's  signed-matched  test 
was  used  to  test  the  hypothesis 
of  no  difference  between  the 
medians  of  the  ISS  and  CISS . 

RESULTS 

The  data  demonstrate  the 
relationships  between  the  number 
of  injuries  and  the  magnitude  of 
the  injury  severity  score  (ISS) 
or  the  complete  ISS  (CISS) 
(Table  4) .  A  majority  of  the 
patients  had  three  or  fewer 
injuries.  For  them,  the  ISS 
and  the  CISS  are  identical, 
because  both  measures  are  based 
on  the  same  most  severe  injuries 
when  there  are  no  more  than 
three  injuries . 


90 


Clearly,  the  CISS  is  higher  than 
the  ISS  when  all  injuries  are 
considered  for  those  individuals 
who  had  FOUR  injuries  or  more. 
The  median  CISS  is  higher  than 
the  median  ISS  by  4.8%,  16%  and 
9.1%  for  individuals  with  4,  5 
and  6  injuries,  respectively. 
Wilcoxon's  signed-matched  test 
indicated  statistically 
significant  differences  between 
the  scores  of  those  individuals 
with  4  and  5  injuries. 
Considering  all  44  patients  who 
had  4  or  more  injuries  yielded 
similar  findings:  the  median  ISS 
was  22.0,  while  the  median  CISS 
was  23.0,  also  a  statistically 
significant  difference. 

When  the  ISS  and  CISS  were 
compared  for  all  761  patients, 
the  differences  were 
statistically  significant.  Thus, 
CISS  is  important  also  in  the 
assessment  of  injury  severity  of 
a  population  with  mixed  patterns 
of  injuries  of  various  severity 
levels. 

DISCUSSION 

The  CISS  is  informative  and 
different  from  the  ISS  only  when 
an  individual  has  four  or  more 
injuries,  and  the  magnitude  of 
the  difference  is  a  function  of 
the  severity  of  the  fourth  and 
more  injuries.  Clearly,  CISS  is 
always  bigger  than  ISS  because 
it  takes  into  consideration  more 
AISs.  In  most  patients,  the 
additional  AISs  that  are 
included  in  the  CISS  calculation 
have  a  smaller  magnitude  than 
those  which  are  included  in  the 
ISS,  because  the  ISS  considers 
the  three  most  serious  AISs. 
Therefore,  CISS  calculations  can 
be  especially  useful  when  four 
of  more  severe  injuries  occur . 
Even  when  the  ISS  serves  as  a 
measure  of  injury  severity,  the 
comparison  of  CISS  and  ISS  is 
suggested  in  order  to  indicate 
whether  there  is   bias  because 
of  unaccounted  injuries  in  the 
ISS.  When  such  bias  exists,  the 
CISS  could  be  used  in  addition 
to  the  ISS.  This  would  be 
appropriate  as   an   analysis 
tool   when  the  examined 
population  includes  many 
patients  with  4  or  more  severe 
injuries. 

An  AIS  6  (unsurvivable)  is  an 
exception  to  all  other  AISs 


because  it  automatically  leads 
to  an  ISS  of  75.  Thus,  the  ISS 
calculations  depend  on  a  single 
injury.  I  suggest  adhering  to 
this  convention,  in  order  to 
enable  comparability  to  the  ISS. 
However,  the  CISS  should  be  used 
when  4  or  more  injuries  are 
considered  and  would  not  be 
useful  in  cases  with  an  AIS  6. 
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Table  2 

Body  regions  for  AIS 

Classification 

1990  revision 

1-  Head 

2-  Face 

3-  Neck 

4-  Thorax 

5-  Abdomen  &  pelvic  contents 

6-  Spine 

7-  Upper  Extremities 

8-  Lower  Extremities 

9-  Unspecified 

Note:  External  injuries  have 
been  dispersed  across 
body  regions. 

Table  3 

Body  regions  for  ISS 
calculations 
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Table  1 

Description  of  the  Abbreviated 
Injury  Severity  Scale  (AIS) 


AIS  =1 
AIS  =2 
AIS  =3 

AIS  =4 


AIS  =5 


AIS  =6 


MINOR 

MODERATE 

SERIOUS  BUT  NOT  LIFE 

THREATENING 

SEVERER,  LIFE 

THREATENING,  SURVIVAL 

PROBABLE 

CRITICAL,  SURVIVAL 

UNCERTAIN 

VIRTUALLY  UNSURVIVABLE 


1-  Head  or  neck 

2-  face 

3-  Thorax 

4 -  Abdomen 

5-  Extremities 

6-  External 


ISS  =  AISi 


Note: 


AIS2' 


AIS3< 


1)  The  ISS  body  regions  do  not 
coincide  with   the   sections 
used  in  the  AIS. 

2)  The  ISS  calculations  included 
head  or  neck  in  one  region 

and  face  in  a  different  ISS 
region . 

3)  The  ISS  calculations  included 
spine  injuries  into  the 
corresponding  three  ISS  body 
regions :  cervical  in  ISS  Head 

or  Neck,  thoracic  in  ISS  Chest 
and  lumbar  in  ISS  Abdominal 
or  Pelvic  Contents . 

Table  4 


The  relationships  between  ISS, 
CISS  and  the  number  of 
injuries.  Median  ISS  and  CISS 


Total 

of 

injuries 

1 

2 

3 

4   5 

6 

N 

440 

197 

80 

33  10 

1 

ISS 

4 

10 

14 

21  25. 

66 

CISS 

4 

10 

14 

22  29 

72 

Wilcoxon 

s 

p  <0.01 

*   * 
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Public  Health 

Applications  using 

Geographic  Information 

Systems 


SURVEILLANCE  OF  CHILD  PEDESTRIAN  INJURIES: 


A  GIS  APPROACH 


Ellen  K.  Cromley,  University  of  Connecticut 
Garry  M.  Lapidus 


The  research  presented  here 
demonstrates  the  usefulness  of 
geographic  information  systems  (GIS) 
technology  for  monitoring  childhood 
injury  arising  from  motor  vehicle 
collisions  with  child  pedestrians. 
Developed  under  the  auspices  of  the 
Connecticut  Childhood  Injury  Prevention 
Center  based  at  Hartford  Hospital  and 
supported  by  the  Hartford  Foundation  for 
Public  Giving  and  the  University  of 
Connecticut  Research  Foundation,  this 
GIS  application  helped  the  Center  to 
determine  whether  and  how  many  high 
frequency  collision  sites  exist  in  the 
Town  of  Hartford  and  what  the  key 
attributes  of  collision  occurring  at 
those  sites  are.   The  databases  compiled 
for  the  project  and  the  GIS  functions 
used  to  analyze  the  data  are  outlined. 
The  implications  of  the  research  for 
designing  public  health  intervention 
programs  to  reduce  childhood  injury  are 
considered,  along  with  several  more 
general  conclusions  about  the  role  of 
GIS  in  public  health. 

Child  pedestrian  injury  is  a  public 
health  problem  of  national  and  local 
importance.   According  to  the  Center  for 
Environmental  Health  and  Injury  Control 
of  the  Centers  for  Disease  Control  and 
Prevention,  pedestrian  injuries  are  a 
significant  cause  of  death  for  all 
children  from  0  to  19  (Center  for 
Environmental  Health  and  Injury  Control 
1990,  11).   Nonfatal  pedestrian  injuries 
are  common  among  5  to  9-year-old 
children,  particularly  males.   An 
estimated  51,000  child  pedestrian 
injuries  occurred  in  the  U.S.  in  1985, 
resulting  in  hospital  admission  of 
18,000  children. 

The  geography  of  motor  vehicle- 
related  deaths  is  captured  in  the  Injury 
Mortality  Atlas  of  the  United  States, 
1979-1987  (Center  for  Environmental 
Health  and  Injury  Control  1991,  29-30). 
Although  most  of  these  deaths  occurred 
among  vehicle  occupants,  pedestrians 
account  for  about  15  percent  of  the 
deaths.   This  common  view  of  the 
geography  of  health  problems  (a  map  of 
rates  calculated  for  county  aggregates 
displayed  with  a  quantile  classification 
scheme)  suggests  that  motor  vehicle- 
related  death  rates  are  lower  on  average 
in  the  northeastern  United  States.   At 
the  local  scale,  however,  we  find  that 
motor  vehicle-child  pedestrian  injury  is 
the  leading  cause  of  injury  death  in  the 
Town  of  Hartford  and  the  second  cause  of 
injury  hospitalization  (Lapidus  and 
Banco  1990).   Recognition  of  this 
problem  prompted  development  of  a  GIS 


application  for  childhood  pedestrian 
injury  surveillance. 

Geographic  information  systems  are 
automated  systems  for  collecting, 
storing,  analyzing,  and  displaying 
spatially-referenced  data  (Department  of 
the  Environment  1987,  132).   For  the 
surveillance  application,  several 
geographic  and  thematic  databases  were 
compiled.   The  geographic  databases  for 
the  project  included  an  address-ranged 
street  network  database  for  the  Town  of 
Hartford  and  a  database  of  1990  census 
tract  boundaries,  both  extracted  from 
the  1990  Census  version  TIGER/Line  Files 
for  Hartford  County  (U.S.  Department  of 
Commerce,  Bureau  of  the  Census  1991). 

The  thematic  databases  included 
road  class  and  average  daily  traffic 
count  data  provided  by  the  Connecticut 
Department  of  Transportation,  1990  child 
population  by  census  tract  from  the  1990 
Census  of  Population  and  Housing  (U.S. 
Department  of  Commerce,  Bureau  of  the 
Census  1992),  and  a  database  of 
information  from  Police  Accident  Reports 
for  all  motor  vehicle  collisions  from 
January,  1988,  through  December,  1990, 
occurring  in  the  Town  of  Hartford  and 
involving  pedestrians  less  than  20  years 
of  age.   These  reports  are  presumed  to 
include  all  serious  injuries  to  children 
but  may  underrepresent  minor  injuries. 
The  State  Department  of  Transportation 
compiles  these  reports  for  all  police 
units  in  the  state  and  provided  a 
printout  of  all  cases. 

The  PAR  database  for  the  study 
period  included  information  on  359 
collisions  spread  almost  evenly  over  the 
three-year  study  period  and  involving 
382  vehicles  and  374  child  pedestrians. 
The  following  information  was  extracted 
from  each  report:   case  number,  date  and 
time  of  collision,  collision  location, 
injury  severity,  environmental 
conditions  (weather,  light,  and  road 
surface),  vehicle  type,  pedestrian/ 
driver  maneuvers,  contributing  factors, 
pedestrian/driver  demographics  (age, 
sex,  and  place  of  residence),  and 
driver/pedestrian  use  of  alcohol  or 
drugs.   Driver  data  were  missing  for 
about  16%  of  drivers  who  left  the 
collision  scene. 

Injury  severity  is  an  obvious 
factor  to  be  considered  in  targeting 
locations  for  public  health 
interventions.   Analysis  of  the  other 
variables  ties  the  injury  to  the  three 
main  prevention  strategies  available-- 
education,  enforcement,  and 
environmental  modif ication--and  research 
on  their  effectiveness. 
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The  specific  GIS  functions  used  in 
the  application  included  compilation  of 
the  geographic  databases,  geocoding 
collision  sites  and  driver  and  child 
pedestrian  residences,  cartographic 
display,  selecting  objects  based  on 
geographic/thematic  attributes,  and 
distance  measurement.   The  geography  of 
motor  vehicle  hazard  is  represented  by 
the  street  network  for  the  Town.   We  can 
modify  this  view  to  represent  segments 
of  the  street  network  where  the  volume 
and  speed  of  traffic  are  high  by 
selecting  those  segments  of  the  street 
network  having  the  specified  attributes 
(Figure  1).   In  this  view,  we  see  the 
locations  of  the  Interstate  Highways  and 
the  principal  and  minor  arterials. 

The  residential  geography  of 
children  in  the  Town  of  Hartford  is 
represented  by  the  census  tract 
distribution  of  the  child  population. 
The  12  highlighted  tracts  in  this  view 
(Figure  2)  account  for  45%  of  the 
population  under  20,  each  tract  having  a 
population  of  at  least  1250  people  in 
this  age  group. 

The  locations  of  motor  vehicle- 
child  pedestrian  collisions,  driver 
residences,  and  child  residences  were 
geocoded  using  a  commercial  GIS  software 
package  (Caliper  Corporation  1990) .   The 
cartographic  display  function  was  used 
to  generate  point  maps  of  collision 
locations  against  the  distribution  of 
child  population  and  against  the  street 
network  (Figure  3). 

Studies  of  collision  clustering 
generally  delimit  linear  segments  of 
fixed  length  in  the  street  network  where 
more  than  a  specified  number  of 
collisions  have  occurred  over  a  fixed 
time  period.   An  examination  of  the 
geographical  distribution  of  collision 
sites  here  uncovered  a  number  of  high 
occurrence  areas  that  could  be 
identified  using  this  approach  (Figure 
4).   One  cluster  occurs  along  a  one-mile 
stretch  of  Albany  Avenue  (federal  route 
44  and  a  principal  arterial)  where  24 
children  were  involved  in  23  collisions 
over  the  three-year  study  period.   A 
second  type  of  "cluster"  is  apparent  in 
occurrences  on  groups  of  adjacent 
streets  having  similar  land-use 
characteristics.   An  example  of  this 
type  of  cluster  occurs  in  the  Park 
Street  neighborhood  where  the  collection 
of  streets  features  a  very  mixed 
commercial/residential  land  use  pattern 
on  Park  and  Broad  (minor  arterials)  and 
their  connectors.   This  network  of 
street  segments,  running  0.8  mile  east- 
west  and  0.3  mile  north-south  accounted 
for  38  collisions  involving  38  children 
over  the  three-year  study  period. 
Together,  the  two  locales  accounted  for 
17%  of  all  collisions,  and  were  selected 
to  illustrate  the  type  of  analysis 
carried  out  in  the  larger  research 
project  (Braddock,  et  al.  1994). 


Once  the  high  collision  locales 
were  identified,  a  profile  of  each 
collision  site  was  prepared  based  on  the 
attributes  of  the  collisions  and  the 
children  and  drivers  involved.   Again, 
the  cartographic  display  functions  of 
the  GIS  were  useful  for  exploring 
differences  between  the  two  sites.   An 
important  variable  for  these  two  locales 
is  the  age  of  the  child  pedestrians 
(Figure  5).   The  average  age  of  child 
pedestrians  struck  on  Albany  Avenue  was 
12  (only  25%  less  than  or  equal  to  7 
years  of  age)  while  the  average  age  in 
Park  Street  was  8  (61%  less  than  or 
equal  to  7  years  of  age). 

Relationships  between  child  and 
driver  residences  and  the  collision  site 
are  also  important  for  intervention 
strategy  development.   The  map  of  the 
residential  locations  of  children 
involved  in  collisions  looks  rather 
similar  to  the  map  displaying  the 
distribution  of  collision  sites.   This 
does  not  necessarily  mean  that  children 
are  being  hit  by  cars  in  their  own 
neighborhoods.   Distance  from  child  and 
driver  residence  to  the  collision  site 
were  calculated  using  the  GIS.   In  the 
Albany  Avenue  case,  for  example, 
children  are  apparently  farther  away 
from  home  when  the  collision  occurs  than 
they  are  in  the  Park  Street  case.   This 
finding  is,  perhaps,  not  surprising 
given  the  age  differences  between  the 
two  groups  of  children.   In  Park  Street, 
however,  the  children  are  often  hit  in 
front  of  their  own  homes  and  it  is 
possible  that  the  difference  in  distance 
to  residence  represents  different  uses 
of  street  space:   as  corridors  for 
movement  in  the  Albany  Avenue  case  or  as 
activity  space  in  the  Park  Street 
neighborhood  case. 

The  Albany  Avenue  collisions  were 
more  likely  than  collisions  at  other 
sites  to  occur  during  the  spring,  in  the 
middle  of  the  afternoon  (around  school 
dismissal  time),  on  a  weekday,  at  an 
intersection,  and  to  involve  older 
children.   The  Park  Street  collisions, 
though  resulting  in  less  severe 
injuries,  were  numerous  and  were  more 
likely  to  occur  around  the  evening  rush 
hour  involving  passenger  cars  that  were 
driving  straight  at  the  time  of  the 
collision  with  a  young  child.   A  site- 
specific  intervention  program  suitable 
for  Albany  Avenue  might  include 
education  programs  for  school-aged 
children;  improved  enforcement  of 
vehicle  speed  laws  and  adherence  to 
traffic  signs;  and  environmental 
modifications  to  improve  intersection 
crossing  times  and  increase  child 
pedestrian  visibility.   In  the  Park 
Street  area,  education  efforts  directed 
at  parents  of  preschoolers  and  young 
school-aged  children,  increased  use  of 
warning  signs,  and  the  development  of 
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off-street  playground  areas  might  be 
effective  prevention  strategies. 

Several  important  conclusions  can 
be  drawn  from  our  experience  using  GIS 
to  study  the  public  health  problem  of 
child  pedestrian  injury.   The  GIS 
analysis  enabled  us  to  identify  high 
occurrence  locales,  to  explore  the 
attributes  of  the  collisions  occurring 
there,  and  to  develop  site-specific 
intervention  strategies.   The  knowledge 
gained  from  this  project  has  enabled  the 
Connecticut  Childhood  Injury  Prevention 
Center  to  work  with  local  groups  to 
design  and  implement  intervention 
strategies  at  a  number  of  the  high 
collision  locales  we  identified 
(Shanahan  1995) . 

Depending  on  the  nature  of  the 
process  being  studied,  "clusters"  of 
health  problems  can  have  very  different 
geometries  (circular  or  linear).   The 
underlying  spatial  structure  of  the 
street  network  or  residential 
development  in  an  area  which  can  be 
represented  effectively  with  GIS 
provides  an  important  context  for 
cluster  identification.   The  GIS  aided 
in  the  delineation  of  substantively 
meaningful  clusters. 

This  application  also  highlights 
the  importance  of  recognizing  that  the 
home  location  is  only  one  site  (though 
an  important  one)  where  human 
populations  can  be  exposed  to  health 
hazards.   Mapping  health  problems  by 
residential  location  may  not  always  be 
appropriate  if  the  home  is  not  where 
exposure  is  occurring.   In  this 
application,  we  were  able  to  represent 
multiple  locations  relevant  to  the 
health  problem.   Related  to  this  point, 
we  realized  the  ability  of  GIS  to 
integrate  databases  from  many  different 
sources . 

The  Injury  Mortality  Atlas  provides 
a  series  of  maps  depicting  the  county- 
level  distribution  of  injury  mortality 
rates  for  various  types  of  injury  for 
the  individual  states.   Based  on  this 
traditional  approach  to  the  geography  of 
health  problems,  the  maps  for 
Connecticut  (Center  for  Environmental 
Health  and  Injury  Control  1991,  48) 
depict  almost  no  geographical  variation 
in  injury  mortality  rates  across  the 
state.   The  application  of  GIS 
technology,  on  the  other  hand,  uncovers 
the  rich  and  complex  geography--worthy 
of  investigation—that  underlies  public 
health  problems  like  child  pedestrian 
injury.   The  development  of  GIS  is 
providing  us  with  a  powerful  new  force 
in  the  conceptual  and  methodological 
armamentarium  of  public  health. 
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Figure  1.   Interstates,  principal  and 
minor  arterials  in  the  street  network, 


Figure  3.   Child  pedestrian  collision 
sites,  1988  through  1990. 


Figure  2.   Census  tracts  where  child 
population  exceeded  1,250  in  1990. 


Figure  4.   Collision  clusters  along 
Albany  Avenue  and  in  Park  Street 
neighborhood . 
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Figure  5.   Locations  of  collisions 
occurring  in  the  clusters  and  involving 
children  7  years  or  younger. 
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Summary 

The  spatia 
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application  of  a 
matched,  vital  s 
Iowa  Department 
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units.  The  signi 
rates  is  determi 
simulation  proce 


1  pattern  of  infant  mortality 
s,  Iowa,  urban  region  is 
ontoured  surface  based  on  the 

spatial  filter  to  address- 
tatistics  records  from  The 
of  Public  Health.   Areas 
g  high  rates  are  shown  to  be 

size  of  the  spatial  filtering 
ficance  of  areas  with  high 
ned  by  a  Monte  Carlo 
dure. 


1.  Methodology 

We  view  infant  mortality  rates  as 
varying  continuously  throughout  any  urban 
area.   Taking  any  location  at  random,  we  can 
define  its  infant  mortality  rate  by  selecting 
an  arbitrarily  sized  region  surrounding  it  of 
a  size  sufficient  to  capture  enough 
observations  of  births  and  deaths  to  estimate, 
reliably,  the  rate.   If  we  repeat  this  for  a 
grid  of  such  estimates,  we  can  interpolate  the 
infant  mortality  rate  as  a  continuous  spatial 
distribution.  From  each,  location  on  a  square 
grid  with  locations  approximately  0.4  miles 
apart,  a  circle  of  0.4  mile  radius  is  drawn 
and  the  number  of  births  and  infant  deaths 
within  the  circle  area  is  computed.   The 
infant  mortality  rate  is  defined  for  the  grid 
location  at  the  center  of  each  circular  area 
as  the  ratio  of  infant  deaths  to  births  found 
in  the  area  for  the  time  period  specified. 
This  method,  known  as  "punctual  kriging"  is 
used  widely  in  the  geo-sciences  (Carrat  and 
Valleron,  1992;  Webster  et  al . ,  1994;  Oliver 
et  al.,  1989).   A  continuous  spatial 
distribution  of  infant  mortality  rates  is 
computed  by  a  contouring  procedure  which 
interpolates  contours  of  equal  rate  values 
from  the  rates  computed  at  the  grid  locations 
(Openshaw  et  al.,  1987;  Openshaw  et  al . ,  1988; 
Turnbull  et  al.,  1990). 

2.  Application 

This  methodology  is  illustrated  in 
Figures  one  through  ten.   In  Figure  1  we  show 
the  spatial  pattern  of  infant  deaths  in  the 
central  part  of  the  Des  Moines  metropolitan 
area  from  1989  through  1992 — the  most  recent 
years  for  which  disease  registry  data  is 
available.   Of  the  209  infant  deaths  in  this 
region  in  this  four  year  period,  192  were 

successfully  address-matched  (Geographic  Data 
Technology,  1993;  U.S.  Bureau  of  the  Census, 
1993).  Their  locations  are  shown  with  their 
locations  (latitude,  longitude  coordinates) 
randomly  distorted  according  to  a  uniform 
distribution  within  an  approximate  0.5  mile 
radius.   This  random  distortion  of  observed 
locations  preserves  the  essential  features  of 
the  pattern,  but  also  preserves  the 
confidentiality  of  the  personal  information, 
which  is  an  absolute  priority  in  studies  of 
disease  distribution,  since  the  rights  of 
individuals  to  privacy  must  be  assured. 

For  the  inset  area  in  Figure  1,  the 
spatial  distributions  of  infant  deaths  and 
births  are  shown  in  Figures  2  and  3.  The 
density  of  births  in  this  region  demonstrates 
that  the  pattern  of  births  for  the  entire 
metropolitan  area  would  be  obscured  if  mapped 


as  a  dot  density  map  at  the  scale  of  Figure  1. 
Figure  4  illustrates  the  spatial  pattern  of 
grid  points  for  the  area  of  Figure  1. 

The  spatial  filter  area  surrounding  each 
of  these  points  is  the  area  from  which  an 
estimate  of  the  infant  mortality  rate  is  made. 
In  this  case,  the  spatial  filter  area  is 
defined  as  within  a  0.4  mile  radius  from  each 
grid  point  and  the  grid  points  themselves  are 
approximately  0.4  miles  apart.  Figure  5  shows 
the  interpolated  infant  mortality  rate  values. 
For  this  area  there  were  at  least  40  births  in 
each  filter  area  between  1989  and  1992.   On 
this  map,  many  small  areas  throughout  the  area 
have  infant  mortality  rates  in  excess  of  20 
per  thousand  live  births  (compared  with  9.5 
per  thousand  for  the  County-wide  rate). 

Details  of  the  spatial  pattern  of  infant 
mortality  are  sensitive  to  the  size  of  the 
spatial  filter.   In  Figures  6  and  7  we 
demonstrate  how  increasing  the  size  of  the 
filter  from  0.4  to  0.8  and  1.2  mile  radius  for 
the  same  grid  points  as  in  Figure  5,  results 
in  a  spatial  pattern  with  fewer  local  high  and 
low  rates  in  the  larger  region.   Locally  high 
rates,  because  they  are  based  on  smaller 
numbers  of  births  and  deaths,  are  likely  to  be 
more  variable  than  regionally  smoothed  rates 
which  are  more  likely  to  reflect  the  region- 
wide  average  rates.   In  the  spatial  statistics 
literature,  the  spatial  filter  is  generally 
referred  to  as  "the  spatial  operator".   Thus 
we  see  in  this  case  that  as  the  size  of  the 
spatial  operator  increases,  the  numbers  of 
areas  with  larger  than  normal  rates  of  infant 
mortality  are  fewer. 

3.  Significance  of  Observed  Rates 

Tests  for  significance  of  these  rates 
were  made  using  methods  described  in  Rushton 
and  Lolonis  (1995).   We  assume  that  the 
probability  a  birth  in  the  area  becomes  an 
infant  death  is  equal  to  the  proportion  of  all 
births  in  the  region  that  resulted  in  infant 
deaths.   A  Monte  Carlo  simulation  is  used  to 
determine  whether  each  birth  location  in  the 
1989-92  period  resulted  in  an  infant  death. 
One  thousand  such  simulations,  resulting  in 
1,000  synthetic  maps  each  of  which  was 
examined  using  the  spatial  grid  and  spatial 
filter  process  described  above.   For  each  grid 
point,  therefore,  1,001  infant  mortality  rates 
existed — the  first  rate  is  the  observed  rate 
and  the  remaining  1,000  rates  are  the 
simulated  rates.   For  each  grid  location,  the 
proportion  of  the  simulated  rates  that  were 
less  than  the  observed  rate  was  computed. 
This  proportion  is  shown  as  a  continuously 
distributed  variable  in  Figure  8  for  infant 
mortality  rates.   The  largest  area  with 
significantly  high  infant  mortality  rates 
(Figure  8)  is  in  the  central  part  of  the  city 
of  Des  Moines  where  a  large  area  exists  in 

which  more  than  75  percent  of  the  simulated 
infant  mortality  rates  were  less  than  the 
observed  rate. 
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Figure  1:  Infant  Deaths,  Dee  Moines,  Iowa, 
1989-1992 


Figure  2:  Infant  Deaths  in  the  Inset  Region  of 
Figure  1 
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Figure  3:  Births  in  the  Inset  Region  of 
Figure  1 


'Hickman  Rd 


I.,.. I     .;■..;.-. 
■ii  ■Cl'apd  Ay     ■''  '•  ■' 


Harllord  Av 


Figure  4:  The  Grid  Locations  for  Which  Infant 
Mortality  Rates  are  Computed 


4.  Conclusions 

Rates  of  infant  mortality  rates  within 
the  Des  Moines  urban  region  vary  markedly 
within  the  region.   The  location  and  extent  of 
high  rates  is  strongly  related  to  the  size  of 
the  spatial  filter  used  to  measure  them. 
Large  spatial  filters  show  few  areas  with  high 
rates.   Smaller  filters  show  more,  smaller 
areas,  with  high  rates.   The  significance  of 
areas  with  high  rates  was  determined  by  Monte 
Carlo  simulations  which  identified  one,  fairly 
large  area,  as  significant. 

Future  work  will  attempt  to  control  for 
known  risk  factors  in  infant  mortality.   With 
respect  to  rates  of  infant  mortality  we  will 
adjust  for  stillbirths,  neonatal  and  perinatal 
death  rates,  as  well  as  known  risk  factors 
such  as  birth  weight,  prematurity,  congenital 
anomalies,  maternal  age,  prenatal  care  and 
race.   Analyses  .that  incorporate  these  factors 
must  be  completed  before  geographically-based 
intervention  policies  to  improve  outcomes  can 
be  determined.   The  Monte  Carlo  simulation 
process,  by  focussing  on  the  locations  of 
individual  births,  can  use  probability  rates 
for  each  birth  location  that  reflect  the  known 
covariates  of  infant  mortality  for  a  birth 
with  the  given  characteristics. 

Could  this  same  conclusion  have  been 
reached  by  using  traditional  sources  (U.S. 
Department  of  Health  and  Human  Services,  1994) 


or  by  following  the  traditional  approach  to 
mapping  infant  mortality  rates  at  the  level  of 
the  census  tract?   Conference  participants  can 
judge  this  by  reviewing  Figures  9  and  10  and 
comparing  the  census  tract  based  patterns  of 
Figure  9  with  Figures  5  and  8.   This 
comparison  shows  that  most  of  the  census  tract 
boundaries  do  not  coincide  with  the  regions  of 
high  infant  mortality  rates.   This  is  clear 
and  persuasive  evidence  that  mapping  health 
data  into  pre-defined  census-defined  areas, — 
or  any  other  exclusively  defined  area  units — 
will  often  conceal  spatial  patterns  that  can 
be  better  discovered  by  more  flexible  tools  of 
geographic  information  analysis.   Such 
exploratory  spatial  data  analysis  methods  are 
useful  for  monitoring  the  health  of  small 
areas  and  defined  population  groups 
( Fotheringham  and  Rogerson,  1994).   The 
approach  has  its  place  alongside  the  more 
traditional  statistical  techniques  of 
confirmatory  analysis  which  are  usually  used 
for  studies  of  suspected  disease  clusters 
(see,  for  example,  U.S.  Department  of  Health 
and  Human  Services,  1990;  American  Journal  of 
Epidemiology,  1990). 

With  support  from  the  U.S.  Department  of 
Education,  we  are  currently  developing  a  CD- 
ROM  which  will  provide  instructional  modules 
on  the  implementation  of  these  methods  as  well 
as  other  applications  of  GIS  and  Public 
Health. 
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Figure  5:  Infant  Mortality  Rates  Defined  by  0.4  Mile  Filter,  Des  Moines,  1989-92 


,A .1 * 

Figure  6:  Infant  Mortality  Rates  Defined  by  0.8  Mile  Filter,  Des  Moines,  1989-92 
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Figure    7:     Infant    Mortality    Rates    Defined    by    1.2  Mile    Filter,    Des    Moines,     1989-92 
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Figure  8:  Significance  of  Infant  Mortality  Rates — Defined  by  Monte  Carlo  Simulation — Des  Moines,  1989-92 
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Figure  9:  Infant  Mortality  Rates  for  Census  Tracts,  Des  Moines,  1989-92 


Figure  10:  Spatial  Overlay  of  Census  Tract  Boundaries  and  Infant  Mortality  Rates 
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There  are  90  million  emergency 
department  (ED)  visits  a  year  in  the  United 
States  (1),  constituting  approximately  3%  of 
US  health  care  costs,  with  an  estimated  $15 
billion  going  to  emergency  department  care  in 
1987.  (2)  Because  charges  for  care  in  the  ED 
are  often  greater  than  charges  for  similar 
care  delivered  in  traditional  primary  care 
settings,  the  increase  in  utilization  of  EDs 
has  generated  substantial  concern.  Therefore  a 
movement  is  underway  to  decrease  inappropriate 
ED  visits.  This  movement  stems  from  two 
motivations :  the  concern  for  cost  containment 
and  the  hypothetical  potential  for  improving 
patient  outcomes  if  channeling  patients  with 
minor  problems  away  from  the  ED  to  traditional 
primary  care  providers  can  enhance  continuity 
of  care. 

However,  implicit  in  this  movement  is 
the  assumption  that  the  phrase  "inappropriate 
emergency  department  visit"  represents  a  valid 
concept  and  that  reliable  measures  exist  to 
ascertain  the  "appropriateness"  of  an  ED 
visit.  That  this  assumption  may  be  problematic 
is  illustrated  by  a  common  clinical  scenario. 
Imagine  a  60  year-old  man  who  develops  chest 
pain.  There  is  nothing  that  a  reasonable  lay 
person  can  be  expected  to  know  or  do  to 
distinguish  the  pain  of  a  life-threatening 
myocardial  infarction  from  a  trivial  cause  of 
chest  pain.  Therefore,  it  is  reasonable  to 
expect  this  patient  to  seek  care  in  an  ED.  In 
the  ED  his  first  contact  will  be  with  the 
triage  nurse,  who  briefly  questions  the 
patient  about  his  symptoms  and  obtains  vital 
signs.  With  this  limited  information,  the 
triage  nurse  cannot  confidently  rule  out  a 
life-threatening  cause  for  the  patient's  chest 
pain.  Then,  the  patient  sees  an  emergency 
physician,  who  obtains  a  more  detailed  history 
and  performs  a  careful  physical  examination. 
The  physician  will  almost  certainly  order  an 
electrocardiogram  and  possibly  other 
diagnostic  tests .  Let  us  suppose  that  after 
this  evaluation  the  emergency  physician 
concludes  that  the  patient  has 

gastroesophageal  reflux  (heartburn) .  Heartburn 
is  clearly  not  life-threatening,  and  the 
patient  is  reassured  and  discharged  home.  In 
retrospect  one  can  argue  that  the  patient's  ED 
care  did  not  improve  his  health,  so  the  ED 
visit  was  not  appropriate.  However 
"appropriateness,"  in  this  sense,  could  not 
have  been  ascertained  prospectively,  prior  to 
a  complete  evaluation  by  the  emergency 
physician. 

Because  of  the  problem  in  measuring 
appropriateness  illustrated  by  this  clinical 
scenario,  we  sought  to  ascertain  whether 
different  measures  of  appropriate  ED  use 
agree.  The  setting  for  the  study  was  the  San 
Francisco  General  Hospital  (SFGH)  ED,  a  high- 
volume  inner-city  ED  with  about  78,000  patient 
visits  a  year.  Patients  waiting  for  emergency 
care  at  SFGH  during  the  week  of  July  9,  1990 
were  eligible  for  inclusion.  Patients  who 
could  not  be  interviewed  were  excluded, 
including  those  younger  than  age  18,  those  who 
did  not  speak  English,  Spanish  or  Cantonese, 
and  those  who  were  not  mentally  coherent.  Also 
excluded  were  those  patients  so  ill  that  they 
were  taken  immediately  to  a  resuscitation 
area.  Finally,  patients  who  left  the  ED  prior 


to  a  complete  evaluation  were  excluded, leaving 
596  patients  for  the  study.  (Not  all  data  were 
available  on  all  patients.) 

Seven  measures  of  appropriateness  were 
utilized  for  the  study.  Two  were  obtained  from 
a  questionnaire  administered  to  patients  in 
the  ED  waiting  room.  Patients  were  asked, 
"Sometimes  people  have  to  wait  several  hours 
in  the  emergency  room.  Instead  of  waiting  now, 
would  you  prefer  to  have  a  doctor's 
appointment  at  a  definite  time  in  one  to  three 
days?"  A  visit  was  classified  as  avoidable  if 
the  patient  indicated  a  preference  for  a 
clinic  visit.  Patients  were  also  asked,  "How 
serious  is  your  medical  problem  now?"  with 
answers  on  a  five-point  scale  ranging  from 
"not  at  all  serious"  to  "extremely  serious."  A 
vi'sit  was  classified  as  avoidable  if  the 
patient  responded  "not  at  all  serious." 

Two  appropriateness  measures  came  from 
the  nursing  triage  sheet.  Triage  nurses  at 
SFGH  categorize  patients  on  a  four-point 
acuity  scale.  Patients  in  triage  category  four 
are  felt  to  be  least  urgently  in  need  of  care 
and  for  this  analysis  were  classified  as 
avoidable  visits.  We  also  studied  a  set  of 
published  triage  guidelines  from  the 
University  of  California  at  Davis,  which 
classify  avoidable  visits  on  the  basis  of 
symptoms  and  vital  signs.  (3-5) 

Three  appropriateness  measures  came  from 
a  review  of  ED  medical  records.  A  visit  was 
classified  as  appropriate  according  to 
previously  described  explicit  criteria  (6)  if 
the  treating  physician  performed  certain 
procedures  or  ordered  specified  therapies  or 
diagnostic  tests.  Second,  an  emergency 
physician,  blinded  to  the  triage  decision, 
rated  the  appropriateness  of  a  random  sample 
of  114  records,  judging  whether  the  patient's 
outcome  might  have  been  worse  if  care  had  been 
delayed  by  24  hours.  Finally,  an  ED  visit  was 
classified  as  appropriate  if  the  visit 
resulted  in  a  hospitalization. 

Analysis  included  calculation  of  the 
proportion  of  ED  visits  deemed  appropriate  by 
each  of  the  seven  measures,  along  with  95% 
confidence  intervals.  In  addition,  each 
possible  pair  of  appropriateness  measures  was 
evaluated  with  respect  to  the  proportion  of  ED 
visits  on  which  both  measures  agreed,  and  the 
percent  agreement  was  adjusted  for  the 
agreement  expected  due  to  chance  alone,  using 
the  kappa  statistic.  (7) 

The  proportion  of  ED  visits  deemed 
appropriate  ranged  from  10%  (using 
hospitalization  as  the  appropriateness 
measure)  to  90%  (using  patient  self- 
assessment)  .  For  the  remaining  five  measures, 
the  proportion  deemed  appropriate  ranged  from 
63%  to  80%,  with  overlapping  95%  confidence 
intervals  suggesting  that  these  five  measures 
deemed  a  similar  proportion  of  visits  to  be 
appropriate. 

However,  when  pairs  of  measures  were 
evaluated  with  respect  to  the  actual  visits 
which  they  deemed  appropriate,  there  was  very 
poor  agreement.  The  proportion  of  visits  on 
which  there  was  agreement  ranged  from  15%  to 
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75%,  with  kappa  statistics  ranging  from  -0.05 
to  0.31.   A  kappa  value  under  0.4  is 
considered  poor  agreement  beyond  that  expected 
due  to  chance .  ( 7 ) 

The  poor  agreement  between  different 
measures  of  appropriateness  raises  questions 
about  the  reliability  and  validity  of  the 
measures.  Although  the  study  was  not  designed 
to  formally  assess  reliability,  the  measures 
were  simple  and  straightforward  and  many  have 
been  used  extensively  in  clinical  contexts  or 
have  been  employed  in  previous  studies.  It 
appears  more  likely  that  the  poor  agreement 
reflects  a  problem  with  validity,  in  that  the 
different  measures  failed  to  measure  the  same 
construct  of  "appropriateness." 

Unfortunately,  these  seven  measures  are 
frequently  used  as  if  they  were 
interchangeable.  This  mistaken  perception  that 
the  measures  are  equivalent  has  important 
implications  for  research,  health  policy,  and 
clinical  practice,  which  will  be  discussed 
here . 

Review  of  the  literature  on 
"inappropriate  use  of  the  emergency 
department"  reveals  substantial  disagreement, 
with  articles  reporting  that  as  few  as  11%  of 
ED  visits  are  inappropriate  (8)  or  as  many  as 
82%  of  ED  visits  are  inappropriate.  (9)  While 
some  of  this  disagreement  may  reflect 
differences  between  the  study  sites,  it 
appears  likely  that  a  substantial  proportion 
of  the  disagreement  is  due  to  inadequate 
methodological  rigor  in  defining  and  measuring 
appropriateness . 

Turning  to  the  health  policy 
implications  of  failing  to  recognize  the 
limitations  of  how  we  measure  "appropriate  use 
of  the  emergency  department,"  the  report  from 
the  1992  National  Hospital  Ambulatory  Medical 
Care  Survey  (NHAMCS)  that  5  5.4%  of  emergency 
department  visits  were  not  urgent  (1)  has 
received  tremendous  media  attention  and  has 
strongly  influenced  policy-makers.  However, 
the  media  and  policy-makers  appear  to  ignore 
the  caution  subsequently  expressed  by  the 
NHAMCS  researchers:  "Hospitals  made  slightly 
different  interpretations  about  how  they 
determined  urgency  for  the  survey.  In  some 
cases,  the  determination  of  urgency  was  based 
upon  the  severity  of  the  patient's  symptom (s); 
in  other  cases,  it  was  based  upon  the 
patient's  diagnosis  or  the  nature  of  the 
treatment  provided."  (10)  In  other  words,  the 
researchers  recognized  that  data  abstractors 
at  the  different  hospitals  in  their  study  had 
used  different  measures  of  appropriateness, 
comparable  to  the  different  measures  which  we 
compared  in  our  study.  One  might  therefore 
expect  variability  in  which  cases  were 
classified  as  appropriate. 

Nevertheless,  policy-makers  continue  to 
discuss  the  construct  of  "inappropriate 
emergency  department  use, "  as  if  the  research 
in  this  area  were  clear-cut.  Two  quotes 
illustrate  this  phenomena,  "Substantial 
Medicaid  savings  could  be  realized  by 
redirecting  non-emergency  visits  to  more 
appropriate  . . .  sites"  (11) ,  and  "Nationwide, 
in  1990,  ...  about  43  million  ED  visits  ... 
could  have  been  treated  in  a  less  expensive 
setting. "  (12 ) 

Clinically,  perception  of  "inappropriate 
emergency  department  visits"  is  manifested  in 
emergency  department  gatekeeping,  the  process 
by  which  managed  care  organizations  require 
pre-authorization  for  emergency  department 
visits.  Because  of  concerns  about  the  clinical 


impact  of  this  practice,  we  have  undertaken  a 
series  of  studies,  all  preliminary  in  nature, 
to  ascertain  the  safety  of  emergency 
department  gatekeeping.  First,  we  solicited 
case  reports  of  adverse  outcomes  occurring 
after  managed  care  patients  presented  to 
emergency  departments  but  were  denied 
authorization  for  ED  visits  by  gatekeepers 
representing  their  managed  care  organization. 
Ten  case  reports  have  been  obtained  to  date. 
These  include  two  adverse  outcomes  occurring 
after  patients  were  denied  authorization  for 
ED  visits  (a  patient  with  meningococcemia  and 
a  patient  with  a  septic  hip) .  One  patient, 
with  epiglottitis,  was  placed  at  increased 
risk  after  care  was  delayed  because  of  a 
denial.  There  were  seven  "near  misses," 
patients  who  were  denied  authorization  for  ED 
visits  but  who  were  seen  anyway  because  of  the 
intervention  of  ED  personnel.  In  each  case,  a 
significant  adverse  outcome  was  judged  to  have 
been  averted  by  the  ED  intervention.  These 
near  misses  included  cases  of  ruptured 
abdominal  aortic  aneurysm,  ectopic  pregnancy, 
cryptococcal  meningitis  in  a  patient  with 
AIDS,  pneumothorax,  incarcerated  inguinal 
hernia,  gastrointestinal  bleed,  and  an 
alcoholic  with  abdominal  pain  and  seizures  who 
subsequently  required  intensive  care  unit 
admission.  (Young  G  and  Lowe  RA,  unpublished 
data) 

This  series  of  case  reports  is  clearly 
limited  by  the  lack  of  denominator  data;  we  do 
not  know  the  frequency  with  which  such  events 
occur.  Therefore,  a  preliminary,  prospective 
cohort  study  was  undertaken  at  the  Hospital  of 
the  University  of  Pennsylvania.  One  hundred 
nineteen  patients  who  were  denied 
authorization  for  ED  visits  by  their  managed 
care  gatekeepers  over  a  three  month  period 
were  included  in  the  study.  Attempts  were  made 
to  contact  all  patients  by  telephone.  When 
patients  could  not  be  reached,  their  primary 
care  physicians  were  contacted  for  follow-up 
information.  Thirty-eight  patients  (32%)  never 
saw  a  physician  after  being  denied  ED  care.  Of 
the  64  patients  we  were  able  to  reach  by 
telephone,  18  (28%)  reported  that  their 
symptoms  were  the  same  or  worse  than  when  they 
sought  care  in  the  emergency  department . 
Thirty  (47%)  were  dissatisfied  with  their 
medical  care.  With  the  telephone  follow-up 
methodology  and  small  sample  size  in  this 
study,  we  did  not  detect  any  patients  with 
adverse  health  outcomes,  however.  (13) 

Our  preliminary  pilot  studies  also 
raised  questions  about  the  equity  of  ED 
gatekeeping.  A  retrospective  study  performed 
in  three  Philadelphia  hospitals  sought  to 
ascertain  which  patients  are  most  likely  to  be 
denied  ED  visits.  Because  the  study  relied  on 
pre-existing  data  at  the  hospitals, 
methodology  varied  between  the  three 
institutions  and  only  partial  adjustment  for 
patient  severity  and  for  the  managed  care 
'organization  to  which  patients  belonged  was 
possible.  Bearing  in  mind  these 
qualifications,  we  observed  the  odds  of  being 
denied  authorization  was  1.6  to  7.2  times 
greater  for  African  Americans  than  for  Whites. 
(14) 

In  summary,  the  concept,  "inappropriate 
emergency  department  visit,"  remains 
undefined.  Results  of  research  using  this 
concept  are  heavily  dependent  on  how  it  is 
measured.  Attempts  to  measure  appropriateness 
of  ED  visits  based  on  current  methodologies 
may  be  hazardous  to  the  validity  of  research 
and  may  lead  to  unsound  health  policy.  There 
is  preliminary  evidence  suggesting  that  misuse 
of  the  concept  of  "ED  appropriateness"  may 
also  be  hazardous  to  patients'  health. 
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Fleming,  Margaret  Hsieh,  Jessica  L.  Cohen  and  Mary  B.  Breckenridge 


Visits  by  children  account  for 
greater  than  one  third  of  all  emergency 
department  (ED)  visits  in  the  US.1 
Unanticipated  revisits  to  EDs  have  been 
used  as  indicators  of  inappropriate  use  of 
EDs,  errors  in  medical  care  and  lack  of 
comprehensive  health  services.  2.3,4.5,6,7,8 
Inappropriate  use  is  expensive,  errors  are 
potentially  dangerous  and  lack  of 
comprehensive  health  care  can  be  both. 
However,  little  is  known  about  revisits  of 
children  to  EDs.  Two  abstracts  describe 
studies  of  revisits  by  children.  However, 
to  pediatric,  not  general,  EDs,  and  one  of 
these  searched  for  reasons  for  revisits  by 
physician  questionnaires  only.910  In 
addition,  insurance  status  has  not  been 
assessed  as  a  risk  factor  for  revisits  in 
any  study  of  revisits  to  date. 

The  aims  of  this  study  were  to 
determine: 

(1)  the  rate  of  revisits  among  children 
cared  for  in  a  general  ED; 

(2)  what  percentage  of  these  revisits  were 
for  related  medical  complaints; 

(3)  the  percentage  of  related  revisits 
that  were  unanticipated  in  that  the 
patient  had  not  been  told  to  follow  up  in 
the  ED;  and 

(4)  any  shared  demographic  and  clinical 
characteristics  among  those  who  revisit 
compared  to  the  characteristics  of  the 
total  population  of  children  who  utilized 
the  study  ED. 

Methods 

The  study  site  was  the  Robert  Wood 
Johnson  University  Hospital  ED  in  New 
Brunswick,  New  Jersey,  a  general  hospital 
that  is  the  primary  teaching  hospital  of 
the  University  of  Medicine  and  Dentistry 
of  New  Jersey  -  Robert  Wood  Johnson 
Medical  School.  This  ED  treats 
approximately  32,000  patients  of  all  ages 
per  annum.  It  is  staffed  24  hours  per  day 
by  physicians  trained  in  emergency 
medicine  and  by  a  pediatric  attending  on 
the  evening  shift.  Residents  in 
pediatrics  and  other  disciplines  are 
available  in  the  hospital  at  all  times. 
Comprehensive  medical  care  is  available  to 
patients  with  public  insurance  or  on  a 
sliding  fee  scale  at  one  of  three  clinics 
located  within  walking  distance  of  the 
hospital.  The  two  largest  of  these 
clinics  provide  evening  appointments  and 
24-hour  telephone  access  to  a  physician. 

The  data  were  collected  from  the  ED 
records  of  all  children  (aged  <18  years) 
who  registered  to  be  seen  in  the  study  ED 


from  July  1,  1992,  to  June  30,  1993.  The 
data  abstracted  include:  age;  race/ 
ethnicity;  gender;  type  of  insurance; 
usual  source  of  health  care;  date  and  time 
of  visit;  presenting  complaint;  type  of 
physician  who  treated  them;  discharge 
diagnosis;  disposition  of  patient;  and 
instructions  for  any  needed  follow  up. 

Data  were  entered  into  the  dBASE  III+ 
database  system.  Discharge  diagnoses  were 
coded  using  the  International 
Classification  of  Health  Problems  in 
Primary  Care  (ICHPPC-2) , "  an  adaption  of 
the  International  Classification  of 
Diseases  9th  revision  for  General  Medicine' 
(ICD-9-GM) ,  devised  with  primary  care  in 
mind. 

Repeated  usage  was  determined  by  hand 
matching  patients  on  the  basis  of  street 
address,  birth  date  and  sex.  This  method 
was  required  by  our  IRB  approval  which 
would  not  allow  the  recording  of  patient 
names  or  medical  record  numbers. 

Revisits  were  defined  as  repeated 
visits  occurring  within  14  days.  A 
related  revisit  was  one  for  a  chief 
complaint  that  was  considered  to  be 
related  to  the  first,  (e.g.,  one  visit  for 
an  upper  respiratory  infection  and  another 
for  otitis  media) .  Unrelated  revisits 
were  those  that  were  unlikely  to  be 
related,  (eg,  one  visit  for  otitis  media 
and  a  second  for  injuries  from  a  motor 
vehicle  accident)  .  An  anticipated  revisit 
was  one  where  the  patient  had  been  told  to 
follow  up  in  the  ED  after  a  defined  time 
interval.  An  unanticipated  revisit  was 
one  where  the  patient  was  told  to  follow 
up  with  their  usual  source  of  health  care 
and/or  local  clinic  or  that  no  follow-up 
was  needed  and  yet  they  returned  to  the 
ED. 

Analyses,  done  with  SAS  (SAS  Insti- 
tute, Cary,  North  Carolina) ,  included: 

(a)  distribution  of  demographic  variables; 

(b)  chi  square  test  of  association  between 
revisits  and  age,  gender,  race/ethnicity, 
and  insurance  status;  (c)  stratified 
analysis  of  these  associations  with 
Mantel-Haenszel  odds  ratios;  and  (d)  odds 
ratios  for  revisits  compared  to  non- 
revisits  for  the  most  common  discharge 
diagnoses. 

Census  tracts  of  addresses  were 
determined  by  matching  with  1990  TIGER 
files  for  the  3  counties  surrounding  the 
hospital  by  the  ATLAS*GIS  mapping  system 
(Strategic  Mapping  Corporation,  Santa 
Clara,  California).  Maps  of  addresses  of 
revisiters  and  total  ED  users  were 
produced  by  Arcview  2.1  (ESRI,  Redlands, 
CA)  . 
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Results: 

For  the  one  year  period,  4276 
children  made  5228  visits.  The  largest 
proportions  of  visits  were  among  those 
under  age  three  (Fig.  1) . 


Figure  1.  Distribution  of  Age  at  Last  Birthday  for 
Children  Presenting  to  ED. 


There  was  a  slight  preponderance  of 
males  (58%  vs  42%)  .  The  largest  racial/ 
ethnic  group  was  white  (41.0%),  followed 
by  black  (33.9%),  hispanic  (18.6%),  and 
asian/other  (6.5%).  More  than  half  of  the 
population  did  not  have  private  insurance 
(31.8%  Medicaid  and  18.3%  none). 

Distribution  of  time  of  presentation 
is  shown  in  Figure  2.  Pediatric 
attendings  saw  49%  of  all  study  children. 


5228  Visits  (4272  Children) 
295  Revisits  (245  Children) 

39  Unrelated  Pairs 


206  Related  Pairs 


/  \ 


37  Expected 


169  Unexpected 


Figure  3.    Distribution  of  Visits  and  Revisits  by  Type 

The  most  common  diagnoses  among  those 
who  revisited  and  who  did  not  are  shown  in 
Table  I  in  rank  order  for  revisits. 

Table  I.  MOST  COMMON  DIAGNOSES 


Diaqnosis 

Revisiters 

Non-revisiters 

(%) 

(%) 

Otitis  Media 

14.1 

9.4 

URI* 

10.4 

7.3 

Lacerations 

6.5 

9.5 

Asthma 

5.5 

3.3 

Viral  Syndrome 

5.1 

3.4 

Gastroenteritis 

3.7 

2.0 

Bronchitis 

3.2 

1.6 

Contusions 

2.9 

6.2 

"\JR\  =  Upper  respiratory  tract  infection) 


Table  II  depicts  the  odds  ratios  of 
revisiting  for  these  diagnoses.  Note  that 
those  who  had  respiratory  diagnoses  were 
more  likely  to  revisit  and  those  with 
minor  trauma  were  less  likely  to  do  so. 


Table  II.  ODDS  RATIOS  OF  REVISITING 
BY  DIAGNOSIS 


Figure  2.  Time  of  Presentation  to  ED 

Revisits  accounted  for  295  of  the 
visits  (5.6%).  Of  the  revisits,  12%  were 
for  unrelated  complaints  (Fig  3) .  Among 
206  related  revisits,  82%  were 
unanticipated.  Closer  examination  of  the 
169  unexpected  revisits  showed  5  children 
had  chronic  diseases  such  as  sickle  cell 
disease  and  leukemia  that  might  be 
expected  to  have  freguent  ED  visits. 
Sixteen  of  the  unexpected  revisits 
involved  a  missed  diagnosis.  Fourteen 
children  left  without  being  seen  one  of 
the  two  times.  This  left  134  children  who 
were  seen  in  the  ED  for  related  diagnoses 
on  two  occasions  within  two  weeks  despite 
the  initial  assessment  that  their 
condition  did  not  warrant  ED  follow  up, 
and  their  return  visit  did  not  reveal  a 
serious  missed  medical  condition. 


Diaqnosis 

OB 

95%  CI 

Bronchitis 

2.10 

1 .65-3.46 

Gastroenteritis 

1.99 

1.18-3.07 

Otitis  media 

1.60 

1.24-2.06 

Asthma 

1.58 

1.13-2.02 

URI 

1.51 

1.13-2.02 

Viral  Syndrome 

1.40 

0.94-2.09 

Lacerations 

0.69 

0.50-0.95 

Contusions 

0.45 

0.27-0.75 

Chi  Square  analysis  reveals  a  strong 
association  between  revisits  and  age 
(p<.0001),  insurance  status  (p<.0001),  and 
race  (p=.007).  No  association  was  found 
with  gender. 


117 


When  revisits  were  analyzed  in  six 
clinically  relevant  age  groups  (0-<2,  2- 
<4,  4-<6,  6-<10,  10-<14  and  14-<18)  with 
6-<10  as  the  referent  group,  Mantel- 
Haenszel  adjusted  odds  ratios  (MHOR)  show 
that  belonging  to  the  age  group  0-<2  was  a 
strong,  independent  risk  factor  for 
revisits  (MHOR  1.67,  CI  1.27-2.19 
controlled  for  insurance;  MHOR  1.89,  CI 
1.47-2.47  controlled  for  race).  MHORs 
with  private  insurance  as  the  referent 
group  indicate  significant  increased  risk 
of  revisits  for  public  insurance  (MHOR 
2.57,  CI  1.93-3.43  controlled  for  age; 
MHOR  2.7,  CI  1.99-3.66  controlled  for 
race) .  MHORs  for  no  insurance,  with 
private  insurance  as  the  referent  group, 
do  not  reveal  any  increased  risk  for 
revisits. 

Of  the  children  who  visited,  87.3% 
resided  in  the  three  counties  for  which  we 
matched  census  tracts  with  addresses.  Of 
the  children  who  revisited,  99.6%  resided 
in  the  matched  counties.  Among  these 
children  we  successfully  matched  90%  of 
the  visit  addresses  and  85.7%  of  the 
revisit  addresses  which  we  could  have 
expected  to  match  (i.e.,  they  had  complete 
street  addresses.) 

Mapping  reveals  a  clustering  of  both 
visits  and  revisits  to  one  area  below  the 
hospital  but  not  to  other  areas  a  similar 
distance  away  nor  to  more  distant  areas 
within  the  municipalities  surrounding  the 
hospital  (Figure  4) . 

Figure  5  is  a  zoomed-in  view  of  the 
area  immediately  around  the  hospital.  The 
area  below  and  to  the  right  of  the 
hospital  includes  5  low  income  census 
tracts.  Per  capita  income  from  the  1990 
census  in  these  tracts  ranges  from  $8,563- 
11,752.  The  number  of  visits  from  this 
area  is  even  greater  than  indicated  by  the 
map  because  many  children  reside  in  a 
housing  project,  and  thus  have  the  same 
address,  so  the  dots  overlap.  An  equal 
distance  above  and  to  the  right  of  the 


hospital  are  three  higher  income  census 
tracts  with  1990  per  capita  income  in  the 
range  of  $16, 848-$24 , 381.  Both  visit 
rates  and  revisit  rates  were  age-adjusted 
by  direct  standardization,  using  as  the 
standard  the  sum  of  the  1990  age 
distribution  for  these  eight  census 
tracts.  The  resulting  visit  rate  for  the 
higher  income  census  tracts  as  a  group  was 
5.7%  and  for  the  lower  income  tracts  as  a 
group  was  14.9%.  The  age-adjusted  revisit 
rate  was  0.2%  for  the  higher  group  and 
1.6%  for  the  lower  income  group. 

Discussion 

The  finding  that  245  children  (almost 
6%  of  the  study  children)  revisited  the  ED 
within  2  weeks  indicates  that  revisits  do 
form  an  important  portion  of  total 
pediatric  ED  visits.  Letourneau  et  al. 
found  2.6%  within  48  hours,  but  this  was 
in  a  pediatric,  not  general  ED. 
Comparison  of  revisit  rates  is  complicated 
by  the  different  time  frameworks  of 
revisits  used.  The  rate  of  revisits  by 
children  found  in  this  study  is  analogous 
to  a  rate  of  5.6%  of  elderly  (>75  years  of 
age)  patients  treated  in  an  ED  who  were 
seen  again  in  the  same  department  within  a 
2  week  period.12  Two  studies  of  all  ages 
in  Detroit  showed  revisit  rates  within  72 


hours  of  3.4%  and  4.0% 


A  Tennessee 


study  of  adults  showed  3%  revisits  within 
48  hours.  Several  studies  in  Great 
Britain  36'7  found  revisit  rates  from  1.9- 
2.5%  across  all  ages  over  varying  time 
periods.  Hu  documented  4.9%  within  one 
week  in  a  veterans'  hospital  in  Taiwan.5 
For  these  studies,  the  different  medical 
system  and  population  many  have  influenced 
the  rates  of  revisits  found.  A  study  of 
all  revisits  to  the  same  institution, 
analyzed  by  age,  would  be  needed  to 
determine  if  children  are  at  particular 
risk  for  revisits. 

The  finding  that  5.4%  of  this  study's 


(a) 


(b) 


Figure  4.  Map  of  municipalities  surrounding  the  hospital  showing  (a)  all  visits  and  (b)  revisits 
from  this  area. 
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(b) 


Figure  5.  Zoomed-in  view  of  area  around  hospital  showing  (a)  all  visits  and  (b)  revisits. 


visit-revisit  pairs  involved  a  missed 
diagnosis  can  be  compared  with  the  rates 
of  9%  found  by  O'Dwyer7  and  11%  found  by 
Keith8  for  the  general  ED  population. 
Keith  found  only  1.4%  of  missed  diagnoses 
in  single  visits  to  their  ED  compared  to 
the  11%  among  revisits,  and  thus  concludes 
that  revisits,  in  particular  unanticipated 
revisits,  form  a  fruitful  area  for  quality 
assurance.  We  did  not  assess  the  overall 
statistic  of  missed  diagnoses  in  our  ED, 
but  agree  they  form  an  appropriate  flag 
for  quality  assurance  efforts. 

Our  finding  that  among  the  five 
diagnoses  with  greatest  risk  of 
revisiting,  four  were  for  respiratory 
illnesses  suggests  that  parents  may  have 
particular  concerns  about  these  illnesses 
and  require  repeated  reassurance. 

Unanticipated  revisits  without  change 
in  diagnosis  were  the  major  proportion  of 
pediatric  revisits  in  this  general  ED. 
The  preponderance  of  unanticipated 
revisits  was  true  for  the  studies  of 
revisits  among  adult  patients  as 
well.2-3-4-5-6'7*  Since  revisits  could 
represent  care  better  delivered  in  other 
settings,  this  deserves  further  research. 

None  of  the  other  available  studies 
of  revisits  have  analyzed  insurance  status 
as  a  risk  factor  for  revisits.  This  is 
partially  because  many  of  these  studies 
are  from  countries  with  national  health 
insurance  (England,  Scotland,  China)  where 
differences  in  insurance  status  would  not 
be  a  relevant  factor.  Those  studies  from 
the  United  States  2AXl°  did  not  report  on 
the  insurance  status  of  the  revisiting 
patients.  Our  finding  that  public 
insurance  is  a  risk  factor  for  revisits  to 
the  ED,  even  when  controlled  for  age  and 
race,  suggests  these  patients  may  be  less 
likely  to  have  a  primary  source  of  health 
care  that  is  available  24  hours  a  day. 
However,  the  local  clinics  do  have  24-hour 
telephone  access  to  a  physician.  Whether 
patients  utilized  this  prior  to  their  ED 
visits  requires  further  research. 

If  lack  of  access  to  a  regular  source 


of  health  care  were  the  only  reason  for  ED 
revisits,  one  would  expect  that  patients 
with  no  insurance  would  also  be  more 
likely  to  revisit  the  ED.  However,  this 
was  not  found  to  be  the  case  in  this 
study,  suggesting  that  the  requirement  of 
paying  out-of-pocket  is  a  disincentive  to 
ED  use.  Our  finding  of  a  markedly  higher 
age-adjusted  revisit  rate  for  the  poorer 
census  tracts  near  the  hospital  than  for 
other  tracts  equidistant  from  the  hospital 
but  of  higher  socioeconomic  means  argues 
against  greater  incidence  of  revisits 
being  simply  a  matter  of  logistics. 

Possible  reasons  for  the  observed 
higher  revisit  risk  of  children  less  than 
two  may  be  that  parents  are  more  concerned 
by  symptoms  such  as  fever  in  younger 
children  than  in  older  children,  and/or 
the  diagnosis  is  more  difficult  in  younger 
children,  leaving  the  parents  less 
convinced  of  the  original  diagnosis  and 
thus  more  likely  to  return  for  a 
reevaluation. 

Conclusions: 

Almost  6%  of  children  who  visit  this 
ED  revisit  within  2  weeks.  Most  of  the 
revisits  are  unanticipated  and  are  related 
to  the  reason  for  the  first  visit.  Most 
revisits  can  not  be  explained  on  the  basis 
of  medical  need.  There  is  a  greater  risk 
of  revisit  for  respiratory  diagnoses  and  a 
lower  risk  for  minor  trauma.  Public 
insurance  is  a  risk  factor  for  revisiting. 
Children  aged  <2  and  residents  of  poorer 
census  tracts  are  more  likely  to  both 
visit  and,  for  some  tracts,  to 
disproportionally  revisit  the  ED.  As  our 
study  was  a  chart  review,  we  were  unable 
to  determine  whether  parents  chose  to 
revisit  the  ED  because  of  lack  of 
understanding  of  discharge  instructions 
for  follow  up,  lack  of  transportation  to 
alternate  sites,  need  for  reassurance,  or 
preferance  for  the  ED  as  a  source  of  care. 
Further  study  is  needed  of  such  parental 
decision  making. 
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One  of  the  first  persons  that 
patients  and  families  encounter  on 
arrival  in  many  emergency  departments  is 
the  triage  nurse.   In  a  comprehensive 
triage  system,  the  triage  nurse  sorts 
patients  by  severity  of  condition  and 
determines  priority  for  therapeutic 
intervention.   Accurate  triaging  can 
decrease  mortality  and  morbidity  and 
lower  the  costs  of  care.   On  the  other 
hand,  over-triaging  less  urgent  patients 
may  waste  expensive  facilities  and 
personnel  and  possibly  delay  the  care  to 
more  seriously  ill  patients.   Under- 
triaging  can  increase  the  risk  of 
patient  mortality  and  morbidity. 

Triage  Acuity  Classification 

The  triage  nurse  must  accurately 
and  rapidly  sort  the  small  numbers  of 
seriously  ill  from  those  individuals 
whose  conditions  are  less  serious  and 
can  tolerate  delays  in  treatment  without 
compromising  safety.   Based  on 
assessment  data,  the  nurse  determines 
the  patient's  acuity  and  assigns  an 
acuity  category. 

While  there  are  many  different 
classification  schemes,  in  general,  the 
acuity  category  conveys  the  degree  to 
which  the  condition  is  life  or  limb 
threatening;  the  risk  for  short  term 
complications;  and  the  availability  of 
treatment  resources  and  providers  (ENA, 
1992) .   For  example,  in  a  three  category 
acuity  scheme  (ENA,  1992)  patients  would 
be  classified  as  emergent  if  their 
condition  was  immediately  life  or  limb 
threatening.   An  urgent  category  would 
be  assigned  to  a  patient  that  required 
prompt  care  but  could  wait  several  hours 
without  threat  to  life  or  limb.   With 
the  non-urgent  patient,  time  is  not  a 
factor.   These  patients  are  able  to  wait 
for  treatment  without  compromising 
safety.   Thus,  the  acuity  category 
reflects  the  seriousness  and  urgency  of 
the  patient's  condition  in  relationship 
to  other  patients  and  available 
resources  in  the  emergency  department. 

Triage  Decision  Making  Study 

The  triage  systems  that  are  used  by 
nurses  have  evolved  primarily  from 
clinical  experience  and  from  knowledge 
gained  from  military  and  disaster 
systems  (ENA,  1992) .   Unfortunately, 
little  research  has  been  done  related  to 
emergency  department  triage.   Therefore, 
we  conducted  a  study  examining  the 
decision  making  of  26  triage  nurses  in  a 
comprehensive  urban  university  medical 
center  emergency  department  that 
provides  services  to  40,000  patients 


annually.   Patient,  nurse,  environ- 
mental, and  task  variables  that 
influenced  triage  decision  making  were 
assessed  in  115  patient  encounters. 

Following  each  triage,  the  nurses 
were  asked  to  describe  their  thinking 
related  to  the  triage  decision  making. 
Qualitative  analysis  of  each  audio-taped 
debriefing  was  completed  by  two 
investigators.   In  addition,  other  data 
were  collected  from  patients'  emergency 
department  records  concerning  the 
patient,  task,  and  environmental 
factors. 

To  determine  the  accuracy  of  the 
triage  acuity  category  assigned,  the 
nurse's  assessment  data,  hypotheses, 
domain  of  information  searched,  triage 
category  selected,  placement  decision 
and  patient  wait  time  were  compared  to 
the  patient's  initial  and  discharge 
medical  diagnoses,  interventions,  and 
discharge  destination.   Consistency  of 
decision  making  among  these  factors  was 
an  indicator  of  accurate  triage.  If 
there  were  inconsistencies  in  the  data 
during  the  course  of  the  ED  visit,  the 
reason  for  the  inconsistency  was 
evaluated.  For  example,  if  the  triage 
acuity  category  assigned  was  unstable 
urgent  but  the  patient  was  not  admitted 
to  the  hospital,  the  inconsistency  would 
have  been  evaluated.   Using  these 
accuracy  criteria,  we  found  that  nurses 
were  highly  accurate  in  their 
prioritization  of  patients  and  their 
assignment  of  the  triage  acuity 
category.   In  two  cases  an  alternative 
acuity  category  was  suggested  by  one  of 
the  coders,  however,  both  coders  agreed 
that  the  category  was  acceptable  as 
used.   Therefore,  based  on  the  study 
emergency  department's  definitions  for 
triage  acuity,  the  nurses  correctly 
separated  patients  into  four  categories: 
emergent,  unstable  urgent,  stable 
urgent,  and  non-urgent  (i.e., 
discrimination) .   In  addition,  the 
nurses  showed  agreement  between 
predicted  and  observed  triage  outcomes, 
(i.e.,  calibration).   In  other  words, 
the  prediction  of  seriousness,  urgency 
and  risk  at  triage  was  consistent  with 
the  seriousness,  urgency,  and  risk  that 
was  observed  during  the  patient's 
emergency  visit. 

Local  Versus  National  Acuity  Data 

Following  triage,  the  nurse's 
acuity  rating  conveyed  valuable 
information  to  the  other  health 
providers  in  the  treatment  area. 
Everyone  in  the  emergency  department 
knew  what  an  unstable  urgent  patient  was 
and  the  resources  that  were  needed  to 
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provide  care  to  this  patient.   The 
triage  acuity  data  in  the  study  ED 
clearly  met  the  needs  of  the  patients 
and  health  providers  in  this  setting. 
Unfortunately,  comparisons  between  this 
excellent  local  triage  acuity  data  and 
other  emergency  departments '  data  cannot 
be  made.   This  local  data  cannot  be 
aggregated  with  other  ED  triage  acuity 
data  to  project  trends  in  health  care 
needs  and  allocation  of  resources.   It 
cannot  be  used  to  guide  decision  making 
for  national  health  standards  and 
policy. 

Inadequate  data.   There  are  several 
reasons  why  triage  acuity  data  cannot  be 
aggregated  and  compared  across  settings. 
In  many  emergency  departments,  there  are 
no  data .   The  Emergency  Nurses 
Association  (ENA)  is  in  the  process  of 
surveying  4,364  emergency  departments 
concerning  their  demographic 
characteristics,  services,  and  issues 
(ENA,  1995) .   The  preliminary  results 
showed  that  14%  of  emergency  departments 
do  not  use  triage  acuity  categories,  18% 
do  not  document  the  acuity  data,  and 
only  5%  document  acuity  on  a  computer. 
Developing  national  statistics  is 
impossible  when  data  are  not  documented 
or  are  documented  but  are  very  difficult 
and  time  consuming  to  retrieve. 

One  solution  to  the  lack  of  data 
will  be  addressed  by  the  development  of 
a  uniform  emergency  department  data  set. 
The  Emergency  Nurses  Association  (ENA) 
is  currently  developing  an  emergency 
nursing  uniform  data  set  that  identifies 
the  essential  and  desirable  data 
elements  that  all  emergency  nurses 
should  collect  on  patients.   The  ENA 
also  is  helping  to  sponsor  and 
participate  in  a  multidisciplinary  group 
led  by  the  Centers  for  Disease  Control 
(CDC)  to  develop  a  national  emergency 
department  uniform  data  set.   In  these 
data  sets,  triage  acuity  classification 
has  been  suggested  as  an  essential  data 
element,  collected  on  all  emergency 
patients,  and  uniformly  defined. 

Definition  variation.   Another 
problem  with  the  current  triage  acuity 
classifications  is  that  emergency 
departments  use  systems  with  different 
categories,  criteria,  and  definitions. 
In  the  preliminary  ENA  survey  data  (ENA, 
1995) ,  about  a  third  of  the  EDs  used  a 
3-category  system,  that  is,  emergent, 
urgent,  and  non-urgent  categories  or 
red,  yellow  and  green  categories.   Other 
systems  included  a  1  to  5  classifica- 
tion, a  1  to  4  classification,  and  a  few 
used  some  other  acuity  system. 
Furthermore,  these  systems  are  not 
defined  in  a  logical  hierarchy  (Anderson 
&  Lees,  1978) .   For  example,  it  is  not 
possible  to  collapse  most  5-category 
systems  down  to  a  3-category  system  for 
statistical  comparisons.   In  summary, 
the  lack  of  data  and  the  variations  in 
triage  acuity  classifications  have 
prevented  the  accumulation  of  data 


concerning  the  acuity  of  patients  in 
emergency  departments  across  the  nation. 
This  lack  of  comparable  data  is  a 
significant  problem.   Health  care  is 
rapidly  changing  and  a  national  database 
of  information  is  needed  for  making 
important  ethical,  economic,  and  health 
care  resource  decisions. 

Research  Directions 

Currently,  there  has  been  no 
methodological  research  done  on  the 
development  and  testing  of  a  triage 
acuity  classification.   However,  several 
authors  have  provided  direction  for  this 
type  of  research  (Brannen,  Godfrey,  & 
Goetter,  1989;  Cowen  &  Kelley,  1994; 
Ruttimann,  1994) .   The  first  priority 
for  research  on  triage  acuity  classifi- 
cation is  to  develop  consensus  on  the 
standard  for  the  acuity  criteria, 
categories,  and  definitions.   Further, 
there  is  a  need  for  research  to  identify 
the  predictor  variables  and  outcome 
states  associated  with  each  of  the 
acuity  categories.   Next,  the  triage 
acuity  system  that  is  developed  needs  to 
be  tested  for  accuracy,  discrimination, 
and  calibration.   Determining 
sensitivity  and  specificity  and  error 
rates  of  the  prediction  rule  should  be 
done.   Because  of  the  risk  to  emergency 
patients,  the  potentially  life- 
threatening  problems  associated  with 
prediction  errors  must  be  assessed. 
Finally,  the  triage  acuity  classifica- 
tion system  needs  to  be  evaluated  on  a 
large,  broad  sample  of  emergency 
patients.   The  results  of  this  research 
program  will  be  a  triage  acuity 
classification  that  is  accurate, 
efficient,  and  effective  for 
communicating  information  concerning  all 
types  of  patients  and  in  all  types  of 
emergency  departments. 

With  a  national  standardized  system 
for  triage  acuity  classification  in 
emergency  departments,  data  will  be 
available  to  guide  health  reform, 
provide  quality  emergency  care,  improve 
health  outcomes,  and  enhance  emergency 
health  services  utilization. 


122 


References 

Anderson,  J.,  &  Lees,  R.  (1978). 
Optional  hierarchy  as  a  means  of 
increasing  the  flexibility  of  a 
morbidity  classification  system. 
Journal   of  Family  Practice,    6,    1271- 
1275. 

Brannen,  A. ,  Godfrey,  L. ,  &  Goetter,  W. 
(1989) .   Prediction  of  outcome  from 
critical  illness:   A  comparison  of 
clinical  judgment  with  a  prediction 
rule.   Arch  Intern  Med,    149,    1083-1086. 

Cowen,  J.,  &  Kelley,  M.  (1994).   Errors 
and  bias  in  using  predictive  scoring 
systems.    Critical   Care  Clinics,    10, 
53-72. 

Emergency  Nurses  Association.  (1992) . 
Triage:  Meeting  the  challenge.  Park 
Ridge,  IL:  Author. 

Emergency  Nurses  Association.  (1995) 
National   Emergency  Department  Database 
Survey.    Park  Ridge,  IL:   Author. 

Ruttimann,  U.  (1994) .   Statistical 
approaches  to  development  and  validation 
of  predictive  instruments.  Critical 
Care   Clinics,    10,    19-35. 


123 


EMERGENT  CARE 
Discussant 


Robert  M.  Williams,  University  of  Michigan  School  of  Public  Health 


Comments  not  available  for  publication. 


124 


Session  H 


Breast 
Cancer 


'    •■•' 


ASSESSING  BREAST  CANCER  RISKS  AND  PREVENTIVE  BEHAVIORS  AMONG  WOMEN  VETERANS 

Denise  Hynes,  Edward  Hines  Jr.  VA  Hospital  and  Loyola  University  Chicago 
Lori  Bastian,  Richard  Sloane,  William  Kalsbeek,  Barbara  Rimer,  and 

John  Feussner 


Cancer  i 
been  estimated 
population  (Ha 
scant  research 
profiles  or  th 
seemingly  high 
no  research  ha 
behavior  patte 
that  will  be  s 
cause  of  death 
This  study  sou 
factor  profile 
patterns  among 
States. 


ncidence  among  women  veterans  has 

to  be  twice  that  of  the  general 
rris  and  Associates,  1985),  yet 

has  examined  the  cancer  risk 
e  health  behavior  of  this 

risk  population.   In  particular, 
s  examined  women  veterans'  risk  or 
rns  for  breast  cancer-a  cancer 
econd  only  to  lung  cancer  as  the 

in  women  in  1995  (ACS,  1995). 
ght  to  assess  breast  cancer  risk 
s  and  preventive  health  behavior 

women  veterans  across  the  United 


Background 

A  number  of  factors  are  known  to  increase 
the  risk  for  breast  cancer  in  the  general 
population,  some  which  can  be  modified  (Willett, 
et  al.,  1987;  Newcomb,  et  al.,  1994;  Hurstin,  et 
al.,  1990;  Hiatt,  et  al.,  1988;  Lowenfels,  1989; 
Longnecker,  et  al.,  1988;  Gail  et  al.,  1989; 
Romieu  et  al.,  1990;  Delgado-Rodriguez,  et  al., 
1991;  Steinberg,  et  al.,  1991;  Dupont  and  Page, 
1991;  Howe,  et  al.,  1990;  Rushton  and  Jones, 
1992;  Silleros-Arenas  et  al.,  1992;  Lemon,  et 
al.,  1992),  and  others  which  allow  targeting  for 
secondary  prevention  (Kelsey,  et  al.,  1991; 
Brisson,  1991;  Mittra  and  MacRae,  1991;  Ewertz, 
et  al.,  1990).   Modifiable  risk  factors  include 
dietary  fat,  excessive  alcohol  consumption  and 
smoking.   Breast  cancer  risk  factors  that  are 
not  amenable  to  primary  prevention,  but  serve  as 
identifiable  characteristics  for  screening  and 
regular  monitoring  include  older  age,  estrogen 
related  factors,  North  American  or  Northern 
European  ancestry,  and  family  history  of  breast 
cancer  (Kelsey  and  Gammon,  1991).   While  a 
consensus  exists  regarding  some  risk  factors  for 
breast  cancer,  there  is  uncertainty  about  other 
factors. 

At  present,  early  detection  is  the  most 
effective  method  of  secondary  prevention.   For 
example,  when  breast  cancer  is  detected  early  in 
stage  I  (i.e.,  tumor  diameter  less  than  1 
centimeter  with  no  cancerous  nodes),  80-90%  of 
women  will  remain  free  of  recurrence 
(Wertheimer,  et  al.,  1986).   Clinical  breast 
examination  and  mammography  have  been  clearly 
established  as  effective  screening  tests  for 
breast  cancer  (Shapiro,  et  al.,  1988;  Tabar,  et 
al.,  1985),  with  mammography  being  the  most 
sensitive  (Health  After  Age  50,  1990).   While 
the  effectiveness  of  screening  programs  may  be 
sensitive  to  the  incidence  of  breast  cancer  in 
the  population  (Nutting,  et  al.,  1994),  studies 
have  shown  that  if  women  followed  mammography 
screening  guidelines,  breast  cancer  deaths 
should  decline  by  at  least  30%  (Tabar,  1985). 
Among  women  aged  50  and  older,  screening 
programs  that  include  regular  mammograms  can 
reduce  mortality  from  breast  cancer  by  as  much 
as  40%  (Tabar,  et  al.,  1985;  Shapiro,  et  al., 
1985).   While  the  benefit  of  mammography  after 
age  50  has  been  conclusively  demonstrated, 
whether  to  initiate  mammography  at  age  40  or  50 
remains  controversial  (ACS,  1988;  NCI,  1987; 
CTFPHE,  1979;  Frame,  1986;  Eddy,  et  al.,  1988; 
Dodd,  1992;  Kerlikowske,  et  al,  1995). 

While  there  has  been  substantial 
improvement  in  the  use  of  mammography  screening 
for  women  in  general  over  the  last  five  years 
(Breen  and  Kessler,  1994),  minority  women  and 
poor  women  still  have  rates  that  are  low  (Fox, 
Siu  &  Stein, 1994;  Perez-Stable,  et  al.,  1994; 
Zavertnik,  et  al.,  1994).  For  example,  in  1991, 


it  was  reported  that  more  than  half  of  women 
over  age  40  had  neither  received  a  mammogram 
(GAO,  Dec,  1991)  nor  were  participating  in 
regular  screening  (Marchant  and  Sutton,  1990). 

The  number  of  women  in  the  military  and 
veteran  population  has  risen  in  the  last  ten 
years.   As  of  1995,  there  were  nearly  200,000 
women  serving  in  the  military  (Office  of  the 
Secretary  of  Defense,  1995),  and  the  number  of 
women  choosing  careers  in  the  military  continues 
to  increase.   Similarly,  the  number  of  female 
veterans  has  increased  in  recent  years,  totaling 
an  estimated  1,234,300  as  of  1991  DVA,  1991; 
Harris  and  Associates,  1985),  with  over  45,000 
female  veterans  receiving  compensation  for 
service-connected  conditions.  Many  of  these 
beneficiaries  are  women  over  50  years  of  age  who 
are  at  greater  risk  of  breast  cancer. 

Methods 

We  surveyed  a  national  sample  of  women 
veterans  by  telephone  to  evaluate  breast  cancer 
risks  and  preventive  behaviors.   This  survey  is 
the  first  phase  of  a  larger  national  survey. 
The  sample  was  selected  from  a  list  of  women 
veterans  constructed  from  databases  from  the 
Defense  Manpower  Data  Center,  the  Internal 
Revenue  Service  and  national  VA  databases  at  the 
Austin  Automation  Center,  and  represents  all 
living  women  veterans  discharged  from  the 
military  between  1971  and  1994.   A  two  phase 
sampling  approach  was  used.   The  first  phase 
sample  comprised  a  10%  sample  of  all  women 
veterans  age  35-70  years  at  the  time  of 
discharge  from  the  military,  who  served  at  least 
18  months  and  were  discharged  between  1971  and 
1994.   20,000  were  sampled  from  the  list  of 
women  veterans  meeting  these  criteria.   A  3x2 
stratification  scheme  was  used:  3  age  groups 
(35-49,50-64,65+)  and  2  VA  user  groups 
indicating  whether  they  received  any  health  care 
from  a  VA  Medical  Center  (VAMC)  in  the  last  five 
years  (VA  user)  or  not  (VA  non-user) . 

The  questionnaire  addressed  breast  cancer 
risk  factors,  including  medical  history, 
environmental  and  occupational  exposures,  diet, 
exercise,  alcohol  and  tobacco  use;  use  of 
mammography  and  other  screening  practices; 
perceptions  about  women's  health  care  at  VA 
medical  centers,  military  service 
characteristics,  and  sociodemographics. 
Validated  items  from  the  Survey  of  Women 
Veterans  and   NCI's  Health  Habits  and  History 
Questionnaire  (HHHQ)  were  included. 

To  locate  women  effectively  and  improve 
response  rates  we  used  aggressive  tracking 
approaches  including  multiple  data  sources  for 
locating  subjects  and  scheduling  call  back  times 
for  conducting  interviews.  98%  of  the  women  were 
located  using  these  tracking  techniques. 
Response  rate  was  75%  (297/397).  Seven  women 
were  excluded  because  they  reported  that  they 
were  not  in  the  military  for  at  least  eighteen 
months,  for  a  total  of  290  interviewed 
respondents. 

All  respondents  were  interviewed  by  a 
female  interviewer.   The  first  100  women 
contacted  were  administered  the  full  survey  (102 
items).   The  remaining  women  contacted  were 
administered  a  brief  15  item  survey  focusing 
only  on  the  breast  cancer  screening  questions 
and  some  basic  demographics.   For  our  analyses, 
the  first  100  responses  were  available  for  the 
risk  factors,  and  all  290  responses  for  the 
breast  cancer  screening  questions  were  used. 


127 


Frequency  of  General  Risk 

Table  1 
Factors  for  Women  Veterans  Compared  to  the  General  Population 

RISK  FACTORS 

WOMEN  VETERANS  (N=100) 

GENERAL  POPULATION  STUDIES 

First  degree  relative  with 
breast  cancer 

12% 

11%  (Gail,  et  al.,  1989) 

Presence  of  benign  disease 

32% 

12-24%  (Gail,  et  al.,  1989; 
Newcomb,  et  al.,  1994) 

Lactation 

57% 

54%  (Newcomb,  et  al.,  1994) 

Table  2 
Frequency  of  Lifestyle  Risk  Factors  for  Women  Veterans  Compa 

red  to  the  General  Population 

RISK  FACTOR 

WOMEN  VETERANS  (N=100) 

GENERAL  POPULATION  STUDIES 

Ever  used  alcohol 

54% 

68%  (Willett,  et  al.,  1987) 

Ever  smoked  cigarettes 

61% 

56%  (Willett,  et  al.,  1987) 

Table  3 
Frequency  of  Risk  Factors  for  Elevated  Estrogens  for  Women  Veterans  Compared  to  the  General 

Population 


RISK  FACTOR 

WOMEN  VETERANS  (N=100) 

GENERAL  POPULATION  STUDIES 

Menarche  before  age  12  years 

17%  (Mean  age  12.8  years) 

15%  (Gail,  et  al.,  1989) 

First  child  at  or  after  age 
30  years 

33% 

11%  (Gail,  et  al.,  1989) 

Never  had  children 

50% 

25-30% 

Late  menopause  (after  age  50) 

21% 

30%  (Newcomb,  et  al.,  1994) 

Table  4 
Frequency  (Weighted)  of  Advice  About  Mammography  Screening  By  Age  Group  (N=290) 

Age  Group 

Advice  Variable 

35-49  Years  (N=135) 

50  Years  and  Older  (N=  155) 

Did  a  doctor  or  a  nurse  ever  discuss 
the  need  to  have  mammography  with 
you? 

72% 

82% 

Did  a  doctor  or  nurse  ever  tell  you 
to  have  a  mammogram? 

66% 

77% 

Frequency  (Weighted)  of 

Table  5 
Use  of  Mammography  By  Age  Group  (N=290) 

Age  Group 

Mammography  Use  Variable 

35-49  Years  (N=135) 

50  Years  and  Older  (N=  155) 

Have  you  ever  had  a  mammogram? 

72% 

92% 

Mammogram  within  the  past  two  years 

47% 

79% 

Predictors  of 

Table  6 
Mammography  Use  Within  the 

Past  Two  Years  (N=290) 

PREDICTORS 

ODDS  RATIO 

CONFIDENCE  INTERVAL 

Age  50  Years  or  greater 

2.52 

1.35-4.67 

Length  of  Service  36-114 
months 

0.86 

0.47-1.59 

Length  of  Service  more  than 
114  months 

2.17 

0.98-4.77 

Black  Race 

0.83 

0.37-1.88 

Used  the  VA  within  the  last 
five  years 

1.58 

0.92-2.70 
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For  purposes  of  this  presentation,  the  35-49 
year  olds  were  compared  to  those  50  years  and 
over,  for  all  analyses. 

Results 

1.  Risk  Factor  Prevalence 

To  put  our  results  in  context,  where 
possible,  we  compare  them  with  published 
literature  on  non-veteran  female  populations. 
Tables  1-3  show  results  for  the  frequencies  of 
specific  risk  factors  among  the  100  women 
veterans  interviewed.   For  example,  Table  1  for 
general  risk  factors  for  breast  cancer,  12/100 
women  veterans  surveyed  in  our  pilot  study 
reported  a   first  degree  relative  with  breast 
cancer  compared  to  11%  in  the  Nurse's  Health 
Study  (Newcomb,  et  al.,  1994). Other  factors, 
although  weaker  than  having  a  family  history  of 
disease,  include  the  presence  of  benign  disease 
and  breast-feeding.   In  our  survey,  more  women 
veterans  reported  having  these  risk  factors  than 
other  published  studies:  32/100  women  veterans 
in  our  pilot  reported  a  presence  of  benign 
disease  compared  to  12-24%  reported  in  other 
studies  (Newcomb,  et  al.,  1994;  Gail,  et  al., 
1989).   Experience  with  lactation,  that  is  ever 
having  breast  fed  an  infant,  was  57%  among  women 
veterans  compared  to  54%  in  the  Nurse's  Health 
Study. 

Table  2  presents  the  lifestyle  factors  for 
women  veterans  compared  to  other  published 
studies.   Of  the  women  veterans  surveyed  54% 
drank  alcoholic  beverages  compared  to  68%  in  the 
Nurse's  Health  Study  (Willett,  et  al.,  1987). 
61%  of  women  veterans  surveyed  had  ever  smoked, 
compared  to  56%  in  the  Nurse's  Health  Study 
(Willett,  et  al.,  1987). 

Table  3  presents  results  for  risk  factors 
related  to  elevated  estrogens.   Of  women 
veterans  surveyed  17%  reported  having  late 
menarche  (mean  age  was  12.8  years  of  age);  33% 
had  their  first  child  after  age  30  and  50%  had 
no  children.   These  rates  were  higher  than  women 
in  other  studies  (Gail  et  al.,  1989;  Newcomb,  et 
al.,  1994). 

2 .  Advice  About  Mammography 

Table  4  presents  results  for  the  290  women 
who  were  asked  about  the  advice  they  received 
regarding  their  personal  need  for  mammography, 
by  age  group.  Among  women  age  35-49,  72% 
reported  that  they  had  a  discussion  with  a  nurse 
or  a  physician  about  the  importance  of 
mammography  and  66%  reported  that  they  were 
instructed  to  have  a  mammogram.  Comparatively, 
women  50  years  and  older  reported  higher  rates: 
82%  had  a  discussion  about  mammography  with  a 
nurse  or  a  doctor  and  77%  were  told  to  have  a 
mammogram. 

3.  Use  of  Mammography 

Table  5  presents  results  for  the  290  women 
respondents  use  of  mammography,  by  age  group. 
Among  women  age  35-49  72%  reported  that  they  had 
ever  had  a  mammogram.  47%  reported  that  they  had 
a  mammogram  within  the  past  two  years. 
Comparatively,  among  women  age  50  years  and 
older,  92%  reported  that  they  had  ever  had  a 
mammogram  and  79%  reported  that  they  had  a 
mammogram  within  the  past  two  years. 

Table  6  presents  results  for  the 
regression  model  for  predicting  mammography  use 
within  the  last  two  years.   Older  women  were  2.5 
times  more  likely  to  have  had  a  mammogram  in  the 
past  two  years  than  younger  women (Odds  Ratio 
2.52,  Confidence  Interval  1.35-4.67).   Women 
veterans  who  had  been  in  the  military  a  long 
time,  that  is,  more  than  114  months,  were  2.1 
times  more  likely  than  women  who  had  been  in  the 
military  less  than  36  months  (O.R.:2.17;  C.I.: 
0.98-4.77).  Whether  a  women  veteran  had  ever 
used  the  VA  for  health  care  was  significant  at 
the  p<0.10  level  and  race  was  not  statistically 
significant.   Only  4%  of  respondents  were  black. 


Discussion 

The  risk  factor  profile  for  women  veterans 
appears  different  than  that  found  in  general 
population  studies.   The  women  veterans  tended 
to  have  higher  frequencies  for  general  risk 
factors  and  for  risk  factors  for  elevated 
estrogens.   Lifestyle  factors  for  women  veterans 
compared  to  general  population  studies  were 
lower  for  alcohol  consumption  but  higher  for 
cigarette  smoking.   As  a  pilot  study,  we 
surveyed  only  100  women  from  across  the  United 
States  about  risk  factors,  and  thus  statistical 
comparisons  are  not  appropriate.   These  trends 
should  be  further  investigated  in  a  larger 
study. 

Regarding  advice  about  mammography,  women 
veterans  are  aware  of  the  need  for  early  breast 
cancer  screening,  but  may  not  always  heed  this 
advice.   In  our  survey  some  women  reported  that 
they  thought  that  mammography  was  only  necessary 
when  a  lump  was  felt  on  clinical  breast 
examination.   There  is  a  clear  need  for 
education  and  outreach  among  women  veterans 
regarding  breast  cancer  prevention  and  early 
detection.   It  is  not  clear  from  this  research 
if  special  efforts  are  needed  to  reach  this 
particular  population,  but  further  study  is 
warranted  to  evaluate  the  best  methods  for 
reaching  women  veterans 

Regarding  use  of  mammography,  women 
veterans  are  reporting  slightly  lower  rates  of 
adherence  than  reported  in  studies  of  non- 
veteran  populations.   In  addition  if  women 
veterans  are  a  higher  risk  population  more 
frequent  screening  may  be  desirable  in  this 
population. 

Regarding  predictors  of  mammography  usage, 
our  results  are  consistent  with  the  notion  that 
women  in  the  military  have  health  screening 
while  in  the  military  and  practice  this  behavior 
after  leaving  the  military.   Further  research 
should  evaluate  not  only  the  length  of  time  in 
the  military,  but  also  the  time  since  discharge 
and  whether  any   breast  cancer  screening 
actually  occurred  when  a  women  was  on  active 
duty  in  the  military. 

Conclusions 

This  study  is  among  the  first  to  examine 
breast  cancer  risk  and  preventive  behaviors 
among  women  veterans.   These  issues  are 
important  because  the  female  veteran  population 
is  aging,  making  them  more  at  risk  for  breast 
cancer,  and  the  female  veteran  population  is 
growing  as  more  women  enter  the  military.   With 
the  growing  number  of  young  women  currently  in 
the  military  and  the  growing  number  of  elderly 
women  veterans,  the  demand  for  women's  health 
care  services  within  the  VA  health  care  system 
in  particular,  will  increase.   Prevention  and 
treatment  for  cancers  affecting  women  veterans — 
most  notably  breast  cancer — will  have  an 
increasingly  significant  impact  on  VA  and  non-VA 
health  care.   Further  research  is  needed  to 
examine  the  needs  and  impacts  of  this 
potentially  high  risk  population  of  women. 
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ABSTRACT 


INTRODUCTION 


The  purpose  of  this  paper  is  to 
illustrate  the  use  of  health  care  data 
for  breast  cancer  prevention.  To 
increase  participation  in  mammography 
screening  programs,  characteristics  of 
women  with  low  utilization  patterns 
need  to  be  identified.  Telephone 
surveys  are  commonly  used  to  assess 
participation  in  screening  programs  but 
their  small  sample  size  and  their  lack 
of  geographic  information  preclude 
analysis  below  the  state  level.  This 
ecologic  study  combines  insurance 
claims  and  census  data  to  analyze 
geographic  variations  in  mammography 
utilization. 

Insurance  claims  data  for  1992  and 
1993  from  3  private  and  4  public 
payors,  covering  close  to  85%  of  the 
state's  female  population,  were 
combined  with  the  1990  Census  ZIP  File. 
It  was  estimated  that  between  30  and 
50%  (median  =  42%)  of  women  40  years 
and  over  had  received  at  least  one 
mammogram  during  the  two-year  period. 
Characteristics  of  geographic  areas 
with  low  utilization  were  identified. 
Results  from  this  study  will  be  useful 
in  targeting  prevention  strategies  and 
surveillance  and  evaluation. 

Figure  1.  Breast  Cancer  Incidence 
and  Mortality,  Hawai 'i,  1986-1990 
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As  in  most  U.S.  states,  breast 
cancer  is  the  most  common  female  cancer 
in  Hawai" i;  close  to  700  cases  (in  situ 
cases  included)  are  diagnosed  annually 
(Hawai' i  Tumor  Registry,  unpublished 
report) .  In  the  absence  of  an  effective 
primary  prevention  strategy,  early 
diagnosis  through  mammography  and 
breast  exam  remain  the  major  approach 
to  decrease  morbidity  and  mortality 
from  breast  cancer.  Whereas 
participation  in  mammography  screening 
has  been  increasing  over  the  last  10 
years1 ,  many  women  have  not  been  making 
use  of  this  early  detection  method. 

Hawai' i's  multiethnic  population 
includes  persons  of  Hawaiian,  Japanese, 
Chinese,  Filipino,  and  Caucasian 
ancestry.  Data  from  the  Hawai' i  Tumor 
Registry  indicate  ethnic  differences  in 
breast  cancer  diagnosis  and  survival 
that  might  be  due  to  differences  in 
screening  and  early  diagnosis.  Compared 
to  the  respective  incidence  rates, 
mortality  rates  are  disproportionately 
high  for  some  ethnic  groups  (figure  1) . 
Survival  after  diagnosis  is  shorter  for 

Figure  2.  Breast  Cancer  Survival  by 
Ethnicity,  Hawaii,  1983-1988 
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Figure   3.    Breast   Cancer, 

Hawai~i,    1989-93,    Percent 

Diagnosed  at   Stage   2    or  Above 


Table   1.    Hawaii's   Health   Insurance,    1993 
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women  with  Hawaiian  and  Filipino 
ancestry  (figure  2),  possibly  because 
more  of  them  are  diagnosed  at  a  later 
stage  (figure  3) . 

To  increase  participation  in 
mammography  screening,  it  is  useful  to 
identify  geographic  areas  or  population 
groups  with  low  participation. 
Mammography  utilization  has  been  mostly 
assessed  through  self-reports,  such  as 
the  Behavioral  Risk  Factor  Survey 
System  (BRFSS)1  with  all  its 
limitations.  Billing  information  from 
insurance  carriers  should  be  able  to 
provide  more  accurate  information. 
Hawai'i  is  an  ideal  place  to  conduct 
research  using  insurance  claims  data 
for  several  reasons.  Because  of  the 
state's  island  nature,  most  of  the 
health  care  is  received  within  the 
state.  Insurance  coverage  is  high,  only 
4%  of  the  population  are  estimated  to 
be  uninsured  (table  1):,  and  the  number 
of  insurance  carriers  is  small. 

Insurance  claims  data  have  the 
advantage  of  being  relatively 
economical  since  they  have  been 
collected  already  and  put  into  an 
electronic  format.  They  cover  large 
segments  of  the  population,  allow 
follow-up,  use  standardized  codes 
(ICD-9  for  diagnosis  and  CPT95  for 
procedures) ,  do  not  need  informed 
consent,  and  are  free  of  recall  bias3,4. 


CATEGORY 

MEMBERS 

PERCENT   NET 
RESIDENT   POP 

PRIVATE   BUSINESS 

HMSA 

604,674 

53.7% 

Kaiser 

187,889 

16.7% 

Commercial 

55,500 

4.9% 

Queen's   Plan 

45,000 

4.0% 

HDS-Medical 

50,500 

4.5% 

Island  Care 

16,662 

1.5% 

TOTAL 

960,225 

GOVERNMENT    PROGRAMS 

SHIP 

21,843 

1.9% 

CHAMPUS 

84, 800 

7.5% 

Medicare 

126,387 

11.2% 

Medicaid 

110,812 

6.8% 

TOTAL 

343,842 

PRIVATE    +    PUBLIC 

1,304, 067 

115.7% 

UNINSURED 

46,201 

4  .1% 

TOTAL    COVERAGE 

1,350,268 

119.8% 

POTENTIAL   OVERLAP 

223,418 

19.8% 

RESIDENT    POP 

1, 126,850 

This  ecologic  study  has  three 
objectives: 

1.  To  determine  mammography 
utilization  rates  for  1992/1993  by 
age  and  geographic  area. 

2.  To  identify  characteristics  of 
areas  with  low  utilization  rates. 

3.  To  illustrate  the  use  of  insurance 
claims  data  for  breast  cancer 
prevention. 

METHODS 

Typical  data  elements  of  an 
insurance  claim  include  patient 
information  (name,  account  &  member 
number,  sex,  birth  date,  zip  code  of 
residence,  and  SSN) ,  encounter 
information  (date  of  service,  up  to  5 
diagnostic  and  procedure  codes, 
charges) ,  and  provider  information 
(provider  ID  and  specialty) .  This 
project  did  not  utilize  any  identifying 
or  financial  information.  Mostly 
scrambled  account  numbers  were  used  to 
track  women  over  the  two  year  period. 
Therefore,  it  was  impossible  to 
identify  any  individuals  and  complete 
confidentiality  was  maintained  at  all 
times.  This  study  was  approved  by  the 
review  boards  overseeing  the  insurance 
carriers.  Data  were  collected  for  an 
estimated  82.5%  of  the  state's 
population.  Information  could  not  be 
obtained  for  the  uninsured,  for  the 
military  population,  and  from  some 
small  commercial  carriers. 
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To  identify  mammographies,  claims  with 

the  following  procedure  codes  were 

extracted  from  the  different  data 

bases: 

8737   Mammography 

40100  Diagnostic  Mammography 

40300  Screening  Mammography 

Z5026  Health  Appraisal  A,  Mammography 

Z5027  Health  Appraisal  B,  Mammography 

Z503  0  Health  Pass  Mammogram 

Z5030  Screening  Mammography 

76090  Diagnostic  Mammogram,  unilateral 

76091  Diagnostic  Mammogram,  bilateral 

76092  Screening  Mammography 

Theoretically,  it  is  possible  to 
separate  diagnostic  from  screening 
mammograms.  However,  it  became  apparent 
early  in  the  project  that  many 
providers  are  using  diagnostic  codes 
for  screening  mammograms.  Until  1991, 
when  insurance  coverage  of  screening 
mammography  was  mandated  by  the 
legislature,  codes  for  diagnostic 
mammograms  were  used  in  order  to  be 
reimbursed  and  not  all  providers  have 
changed  their  coding  practice  since 
then.  Therefore,  all  mammograms 
regardless  of  coding  are  included  in 
this  study. 

Data  processing  included  deleting 
duplicates  and  claims  with  invalid  ZIP 
codes,  combining  data  sets  from  the 
different  private  and  public  payors, 
keeping  one  claim  per  women  per  year 
(any  additional  would  not  be  screening 
mammograms) ,  producing  frequency  tables 
by  ZIP  code,  and  combining  data  sets 
with  the  1990  ZIP  census  file. 
Approximately  3.5%  of  claims  had  ZIP 
codes  designating  post  office  boxes 
rather  than  geographic  areas.  If 
possible  these  claims  were  assigned  the 
ZIP  code  of  the  geographic  area  that 
includes  the  particular  post  office. 
Utilization  rates  were  calculated  by 
dividing  the  number  of  women  with  at 
least  one  mammogram  during  1992/93  by 
the  estimated  number  of  women  living  in 
the  geographic  area. 


As  denominator,  the  1990  census 
ZIP  file  on  CD-ROM  as  distributed  by 
the  Census  Bureau  was  used.  It  provided 
information  on  the  number  of  women  by 
age,  ethnic  distribution,  income,  and 
educational  achievement.  Since 
intercensal  estimates  are  not  available 
for  individual  ZIP  code  areas,  the 
population  size  was  increased  by  the 
statewide  population  growth  rate  for 
women  4  0  years  and  over  as  published 
annually  by  the  Census  Bureau. 


Table  2 .  Data  Sources  for 
Mammography  Claims , 
Hawai  i,  1992-93 


Payor 

Private  1 

Private  2 

Kaiser 

Medicare 

Medicaid 

CHAMPUS 

Total 


1992 

33, 932 
1,254 

13, 678 

10,292 

1,799 

510 

61,465 


1993 

37, 335 

1,  139 

18,077 

9,819 

1,  997 

407 

68,774 


Multiple  linear  regression  using 
the  stepwise  method  was  utilized  to 
develop  models  with  mammography 
utilization  rate  as  dependent  variable 
and  demographic  information  and 
insurance  membership  as  independent 
variables.  Variables  with  a  non-normal 
distribution  were  transformed  by  an 
appropriate  operation.  Data  management 
and  analysis  were  performed  with  PC- 
SAS®  (SAS  Institute,  Cary,  NC) . 

RESULTS 

A  total  of  61,465  and  68,774 
mammography  claims  for  women  4  0  years 
and  over  were  obtained  for  1992  and 
1993  respectively  (table  2) ,  after 
deleting  repeat  mammograms  during  the 
same  year.  The  median  mammography 
utilization  rate  is  42%.  After 
weighting  by  population  size,  most  ZIP 
code  areas  have  a  utilization  rate 
between  30  and  50%  (figure  4) .  Rates 
differ  by  age  group  but  very  little  by 
year  (figure  5).  26%  of  all  women  had 
more  than  one  mammogram  during  the  two 
year  period.  This  percentage  was  higher 
for  women  between  50  and  64  years  (32%) 
than  for  women  65  years  and  over  (25%) 
and  women  40  to  49  years  (18%) .  Mapping 
the  utilization  rates  illustrates 
geographic  areas  with  high  and  low 
utilization  and  will  be  useful  for 
future  interventions  (figure  6). 

A  comparison  of  self-reported 
mammography  utilization  rates  from  the 
Behavioral  Risk  Factor  Survey  System 
with  results  of  this  study  (figure  7) , 
suggests  that  self-reports  are  at  least 
20%  higher  than  estimates  from 
insurance  claims  data. 
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Figure  4 .  Mammography 

Utilization  by  ZIP  Code  Area, 

Hawai"i,  19  92-93 


Figure  5 .  Mammography  Utilization 

Rates  by  Year  and  Age, 

Hawai'i,  1992-93 
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The  best  regression  model  with 
only  demographic  variables  explains  24% 
of  the  variance.  It  includes  low  income 
(<$30,000  household  income  per  year), 
percent  Japanese  in  the  population,  and 
urban  environment.  A  combined  model 
explains  56%  of  the  variance.  It 


40  -  49  yrs 


64  yrs 


65  -t-  yrs 


variance.  It  includes  Blue  Cross  Blue 
Shield  coverage,  low  income,  urban 
environment,  and  Kaiser  coverage.  Age, 
county  of  residence,  percentage  of 
other  ethnic  groups,  the  presence  of  a 
mammography  unit  in  ZIP  code  area,  and 
educational  achievement  are  not 
significant  in  any  of  the  regression 
models. 


Figure   6.    Mammography  Utilization  Rate 

By  Zip  Code,  Oahu,  Hawai'i,  1992-93 
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Figure  7.  Mammography  Utilization, 

Hawai~i, 1992/93 
Comparison  of  BRFSS  and  Claims  Data 
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DISCUSSION 

This  study  estimates  that  at  least 
42%  of  women  4  0  years  and  over  had  a 
minimum  of  one  mammogram  during 
1992/93.  Adjusting  for  the  unavailable 
data,  the  percentage  may  be  as  high  as 
50%.  In  the  BRFSS,  71%  of  women  4  0 
years  and  over  reported  a  mammogram 
during  the  most  recent  two  years.  This 
overestimate  is  probably  due  to 
selection  and  recall  bias  in  the 
telephone  survey.  The  geographic 
variation  in  utilization  rates  was 
surprisingly  small.  Demographic 
information  from  the  census  file  was 
not  a  good  predictor  of  utilization 
rates.  Geographic  areas  with  low 
mammography  utilization  were  more 
likely  to  have  low  Blue  Cross  and 
Kaiser  coverage,  a  high  percentage  of 
households  with  low  income  (<$30,000), 
a  low  percentage  of  persons  with 
Japanese  ancestry,  and  to  be  in  an 
urban  environment. 

The  major  limitation  of  this  study 
is  that  data  are  unavailable  for  17%  of 
the  population,  mostly  for  the  military 
and  the  uninsured  population  and  for 
persons  covered  by  small  private 
carriers  who  are  unable  or  unwilling  to 
share  their  data.  The  lack  of  a  unigue 
identification  number  is  another 


serious  problem  because  it  makes  it 
impossible  to  guantify  dual  insurance 
coverage,  estimated  at  19%  in  Hawaii, 
and  to  track  individuals  when  they 
change  health  plans.  Accurate 
denominators  for  intercensal  years  on 
ZIP  code  level  would  also  be  desirable. 
Insurance  claims  do  not  contain 
information  on  ethnicity  which  makes  it 
impossible  to  calculate  ethnic-specific 
utilization  rates.  Ecologic  models 
using  the  percentage  of  certain  ethnic 
groups  by  ZIP  code  as  done  here  cannot 
assess  ethnic  differences  in 
mammography  utilization  with  great 
accuracy. 

In  the  future,  the  results  of  this 
study  will  be  linked  to  breast  cancer 
incidence  and  mortality  data  on  a 
geographic  level.  This  data  set  will  be 
useful  to  monitor  time  trends  in 
mammography  utilization  and  as  a 
surveillance  tool  to  evaluate 
interventions  to  increase  participation 
in  mammography  screening.  Insurance 
claims  have  great  potential  in  the  area 
of  cancer  prevention  mainly  because 
they  allow  assessment  of  participation 
in  cancer  screening  for  population 
subgroups  and  specific  geographic 
areas. 
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INTRODUCTION 

Breast  cancer  is  one  of  the  most 
frequent  malignancies  affecting  women  today. 
It  is  the  most  common  cause  of  death  due  to 
cancer  in  females  ages  40-50  (Eddy, 1989).   In 
1991,  the  American  Cancer  Society  estimated 
175,000  new  cases  of  invasive  breast  cancer 
diagnosed  in  females .   The  most  important 
tools  in  the  early  detection  of  breast  cancer 
are  the  clinical  breast  exam  and  screening 
mammography.   Research  indicates  that 
mortality  due  to  breast  cancer  can  be  reduced 
by  3  0%  among  women  aged  50  and  over  through 
the  use  of  these  simple  screening  methods 
(Shapiro,  1977;  Tabar,  1985)  .  One  study  noted 
that  the  average  sensitivity  of  the  combined 
clinical  examination  and  mammography  over  a 
five-year  time  period  was  75%  while  the 
specificity  of  mammography  approached  99%.  (US 
Preventive  Services  Task  Force,  1989).   Yet 
the  number  of  women  receiving  these  preventive 
services  remains  alarmingly  low.   In  the  1987 
National  Health  Interview  Survey  Cancer 
Control  Supplement,  only  36%  of  women  aged  40 
and  over  had  EVER  received  a  clinical  breast 
exam  and  screening  mammography.   For  those 
women  age  50  and  older  only  25%  had  reported 
having  had  a  clinical  breast  exam  and 
screening  mammogram  within  the  preceding  2 
years  (Public  Health  Service,  1990,  page  429). 

In  1990  the  Public  Health  Service  in 
Healthy  People  2000:  National  Health  Promotion 
and  Disease  Prevention  Objectives  established 
the  goal  of  increasing  to  at  least  80  percent 
the  proportion  of  women  aged  40  and  older  who 
have  ever  received  a  clinical  breast 
examination  and  mammogram,  and  to  at  least  60 
percent  those  aged  50  and  older  who  have 
received  them  within  the  preceding  1  to  2 
years.   Since  the  Public  Health  Service 
established  this  goal,  screening  activities 
have  increased  to  64%  of  women  over  40 
reporting  EVER  having  had  a  mammogram  in  1990 
(NCQA,  1993)  . 

In  an  effort  to  improve  the  process  for 
the  evaluation  of  patients  with  breast 
disease,  Mayo  Clinic  is  currently  implementing 
the  breast  cancer  screening  and  diagnosis 
practice  guidelines.   These  guidelines  were 
developed  with  the  Institute  for  Clinical 
Systems  Integration  (ICSI) .    ICSI  was  founded 
in  1993  through  the  joint  efforts  of  the  Mayo 
Clinic,  Group  Health,  Park  Nicollet  Medical 
Center  and  HealthPartners  as  a  response  to  the 
Business  Health  Care  Action  Group,  a  coalition 
of  24  Minnesota  businesses.   ICSI  provides 
health  care  quality  improvement  services  to  20 
medical  groups  affiliated  with  HealthPartners 
in  the  Minnesota  and  Wisconsin  region  using 
continuous  quality  improvement  principles  to 
standardize  health  care   processes  and  to 
improve  health  care  outcomes.   Health  care 
guidelines  represent  a  process  for  treatment 


or  prevention  of  specific  health  conditions. 
The  continuous  quality  improvement  cycle  is 
used  to  manage  guideline  development, 
implementation  and  improvement.   To  date,  26 
guidelines  have  been  approved  for 
implementation  by  participating  medical 
groups . 

The  first  component  in  the  breast  cancer 
screening  and  diagnosis  practice  guidelines  to 
be  implemented  in  the  primary  care  divisions 
at  Mayo  is  the  Primary  Care  Evaluation  of  the 
Breast  (ICSI,  1995) .   This  process  starts  with 
asymptomatic  patients  undergoing  a  screening 
examination,  or  symptomatic  patients 
presenting  with  breast  related  complaints. 
The  process  ends  when  a  diagnosis  is 
established  either  by  biopsy  or  by  clinical 
and  radiological  findings.   By  reducing  the 
variability  of  the  evaluation  process,  we  hope 
to  improve  the  accuracy,  timeliness  and  cost 
of  care.   With  compliance,  through  increased 
screening  mammography  rates,  we  hope  to 
identify  breast  cancer  at  its  earliest 
possible  stage  and  improve  survival  for 
patients  with  breast  disease. 

Performance  and  compliance  with  the 
guideline  are  assessed  through  standardized 
reporting  measurements  which  have  been  defined 
in  the  development  process .   The  measurement 
goals  for  the  breast  disease  guidelines  are  to 
identify  the  stage  of  cancer  at  diagnosis  and 
to  accurately  report  screening  rates . 

The  mammography  screening  intervals 
recommended  by  the  guideline  include:  a 
monthly  self  breast  examination  beginning  at 
age  3  0;  a  professional  breast  examination 
every  3  years  for  those  aged  20-39  and 
annually  for  those  aged  40-75.   For  those 
women  with  high  risk  for  breast  cancer, 
mammography  is  recommended  annually  beginning 
at  age  40,  with  a  baseline  measurement  done 
between  the  ages  of  35-40.   For  those  women  at 
low  risk,  mammography  is  recommended  annually 
beginning  at  age  50  and  is  optional  for  women 
age  40-49.   A  baseline  measurement  is 
recommended  at  age  40  for  those  women  at  low 
risk.   Given  the  lack  of  consensus  regarding 
the  beneficial  affect  of  screening  mammography 
for  women  age  40-50  (Feig,  1994;  Sickles, 
1995;  Smart,  1995),  the  breast   mass  guideline 
group  considers  the  use  of  mammography  to  be 
OPTIONAL  in  this  age  group.   Its  use  is  left 
to  the  discretion  of  the  primary  care  provider 
and  the  patient.   Yearly  mammography  is 
appropriate  for  women  in  this  age  group  found 
to  be  at  high  risk. 

The  basic  principles  and  general 
guidelines  for  breast  cancer  screening  defines 
the  target  population  as  women  age  50-74 
continuously  enrolled  in  a  health  plan  for 
twelve  months .  The  three  developed  measures 
deal  with  mammography  screening  among  all 
enrolled  women;  enrolled  women  who  had  a 
primary  care  visit;  and  enrolled  women  who  had 
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a  pelvic  exam  and  pap  smear  or  a  complete 
physical  examination. 

In  addition  to  screening  rates,  the 
guideline  measures  address  outcome  by 
examining  the  stage  of  cancer  for  all  patients 
diagnosed.   The  basic  principles  and  general 
guidelines  for  breast  cancer  diagnosis  defines 
the  target  population  as  all  newly  diagnosed 
women  with  breast  cancer  through  age  74.   The 
two  developed  measures  are  defined  as  the 
proportion  of  early  stage  breast  cancer  at 
initial  diagnosis  (early  stage  is  defined  as 
stages  0,1  or  2.)  and  the  percent  of  women  at 
stage  2. 

METHODS 

We  have  evaluated  baseline  breast  disease 
guideline  measurements  among  patients  served 
by  seven  sites  within  Mayo.   Mayo  Clinic  is 
located  in  southeastern  Minnesota  in  the  city 
of  Rochester.   Although  best  known  for  its 
large  referral  and  specialty  practice,  Mayo 
also  has  an  extensive  primary  care  practice. 
The  seven  sites  evaluated  include  five  primary 
care  sections,  the  urgent  care  center  and  the 
specialty  division  of  gynecology  (GYN) .   The 
five  primary  care  sections  include  two  family 
practice  sites:  the  Kasson  Clinic  and 
Rochester  Family  Practice;  and  three  internal 
medicine  sites:  Area  Medicine,  Community 
Internal  Medicine,  and  Preventive  Medicine. 
The  demographics  of  the  patients  each  division 
serves  varies  as  well  as  its  location  within 
the  Mayo  complex.    All  of  the  evaluated  sites 
are  located  on  the  Mayo  Rochester  campus  with 
the  exception  of  the  Kasson  Clinic.   The 
Kasson  Clinic  is  a  small  clinic  located  15 
miles  to  the  west  of  Rochester  in  Kasson, 
Minnesota.   The  Kasson  Clinic  serves  mainly  a 
rural  population.  The  Preventive  Medicine 
patient  population  is  primarily  Mayo  Clinic 
employees.   The  Urgent  Care  Center  is  a  walk- 
in  clinic  which  targets  patients  with  acute 
conditions,  and  is  staffed  by  physicians  from 
family  practice  and  internal  medicine  on  a 
rotating  basis.   The  last  group  we  examined  is 
GYN.   This  division  is  staffed  by  specialists. 

Mayo  clinic  has  adapted  the  ICSI  breast 
cancer  screening  guideline  measurements  to  its 
fee-for-service  practice  using  our  1993 
billing  files.    However,  not  all  of  the 
measurements  defined  by  ICSI  were  applicable 
in  our  setting.   Since  Mayo  patients  are  not 
continuously  enrolled  in  the  system,  all 
patients  with  Mayo  services  provided  in  the 
calendar  year  have  been  considered.   Patient 
date  of  birth  and  gender  have  been  extracted 
from  the  last  Mayo  registration  in  1993.   A 
patient's  experience  for  the  entire  year  was 
assessed.   Therefore,  the  measurements  from 
the  breast  cancer  screening  guideline 
have  been  adapted  to  the  Mayo  system  and 
defined  as  mammography  screening  rates  among 
women  aged  50-74  with  ANY  service  provided  in 
the  calendar  year.   The  number  of  patients 
with  a  mammogram  has  been  defined  as  any 
patient  with  CPT  codes  76090-76092.   The 
historical  database  does  not  identify  the 
physician  or  division  ordering  services. 


Therefore,  for  the  accompanying  tables,  a 
patient  was  assigned  to  each  division  where 
they  were  seen  in  the  calendar  year.   In  this 
way,  an  individual  patient  may  be  reported  in 
one  or  several  divisions.   For  a  patient  seen 
during  the  year  in  both  the  divisions  of 
Family  Practice  and  GYN,  who  had  a  mammogram 
ordered  by  GYN,   Family  Practice  would  also 
receive  credit  for  this  service.   It  is 
important  to  keep  in  mind  that  in  our  study 
the  mammography  rates  are  encounter  based,  not 
population  based. 

The  guideline  measurement  for  breast 
cancer  diagnosis  did  not  require  modification 
to  the  Mayo  system.  Newly  diagnosed  cases  of 
breast  cancer  at  Mayo  were  identified  through 
the  cancer  registry.  American  Joint  Committee 
on  Cancer  (AJCC)  criteria  were  used  to  assess 
the  tumor  stage  at  the  time  of  diagnosis. 

RESULTS 

The  baseline  mammography  screening  rates 
for  females  ages  50-74  varied  substantially 
across  the  practice  groups  from  49%  in  the 
rural   family  practice  (Kasson  Clinic)  to  76% 
in  Preventive  Medicine  (Table  1) . 
Not  surprisingly,  patients  seen  for  preventive 
care  or  for  pelvic  examinations  had  higher 
mammography  screening  rates  than  those  women 
who  did  not  receive  this  preventive  care  in 
the  calendar  year.   Overall,  women  who  did  not 
receive  a  general  medical  or  pelvic  exam,  in 
the  calendar  year,  had  a  screening  rate  of 
40%,  but  the  rate  increased  to  74%  for  those 
women  with  these  preventive  services.   The 
screening  rates  among  women  with  preventive 
exams  ranged  from  66-81%,  while  the  rates  for 
women  without  these  services  ranged  from  22- 
55%.   Much  of  the  higher  rates  for  Preventive 
Medicine  and  Area  Medicine  is  explained  by  the 
predominance  of  patients  with  pelvic  and 
general  exams.   The  higher  proportion  of  acute 
care  patients  in  the  Urgent  Care  Center  and 
the  rural  family  practice  (Kasson  Clinic) 
resulted  in  lower  overall  mammography  rates. 

Since  Medicare  reimbursement  for  a 
mammogram  is  every  2  years,  it  was  anticipated 
that  the  mammography  rate  may  be  lower  for 
those  women  over  age  65  in  our  fee-for-service 
practice.   However,  this  was  not  the  case. 
Overall  screening  rates  varied  only  slightly 
by  age,  65%  for  those  age  50-54,  67%  for  those 
age  55-59,  66%  for  those  age  60-64,  67%  for 
those  age  65-69  and  62%  for  those  age  70-74 
(Figure  1) . 

In  addition  to  screening  rates,  the 
guideline  measures  address  outcome  by 
examining  the  stage  of  cancer  for  all  patients 
diagnosed.   In  1993  there  were  399  newly 
diagnosed  cases  of  breast  cancer  at  Mayo 
among  women.   Ages  ranged  from  29-94.   Ninety- 
one  percent  were  diagnosed  in  an  early  stage 
(stage  0-2)  (table  2) .   The  ICSI  defined 
measurements  include  only  those  through  age 
74.   Excluding  those  age  75  and  older  the 
overall  rate  of  those  diagnosed  in  early  stage 
is  89%.   Given  the  considerable  controversy 
surrounding  the  beneficial  effects  of 
screening  mammography  for  women  age  40-49,  we 
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Table  1:   1993  Rates  by  Division  for  Females  Ages  50-74 


Mammography  Screening 
Rates  (%) 

Preventive  Care 

(%) 

'Overall 

-  Preventive 
Care 

+  Preventive 
Care 

Overall 

Rural  Family  Practice 

49 

27 

66 

57 

Rochester  Family  Practice 

65 

38 

76 

73 

Community  Internal  Med 

74 

54 

81 

75 

Area 

73 

41 

78 

87 

Preventive 

76 

45 

81 

86 

Urgent  Care  Center 

53 

22 

72 

63 

OB/GYN 

67 

55 

69 

80 

Overall 

65 

40 

74 

-- 

Figure  1:   Population  vs  Fee-For-Service  Mammography  Screening  Rates,  1993 
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Table  2:   Women  Diagnosed  at  Mayo  With  Breast  Cancer,  1993 


All  Women 

Those  at  Assessed  Sites 

N 

Stage  (%) 

Lymph  node 
involvement  % 

N 

Stage  (%) 

Lymph  node 
involvement  % 

0-2 

2 

0-2 

2 

9 

89 

33 

11 

7 

86 

43 

14 

40-49 

4'i 

86 

44 

37 

14 

79 

36 

50 

50-74 

256 

90 

29 

25 

103 

96 

29 

24 

75  + 

91 

96 

29 

17 

35 

94 

31 

23 

Overall 

399 

91 

30 

24 

159 

94 

31 

26 
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have  Ick    I  at  1  his  data  in  the  following  age 
groups;  <40,  40-49,  50-74,  and  75+.    The 
percent  of  those  women  diagnosed  in  an  early 
stage  was  somewhat  stable  across  these  age 
groupings,  ranging  from  86%  for  those  aged  40- 
49  to  96%  for  those  aged  75  and  older.   The 
percent  of  these  patients  with  stage  2  breast 
cancer  and  lymph  node  involvement  at  the  time 
of  diagnosis  were  more  varied  across  the  age 
range.   Forty-four  percent  of  those  patients 
aged  40-49  compared  with  about  30%  for  the 
other  age  groups  were  diagnosed  at  stage  2. 
Those  patients  ages  40-49  also  had  a  higher 
percentage  of  patients  with  lymph  node 
involvement  at  the  time  of  diagnosis  (37%). 
Overall  24%  of  all  newly  diagnosed  cases  of 
breast  cancer  had  some  lymph  node  involvement 
at  the  time  of  diagnosis. 

Examining  all  cancers  newly  diagnosed  at 
Mayo,  we  include  many  patients  who  have  been 
referred  to  us  for  specialty  care.   It  should 
be  noted  that  61%  of  those  patients  with  a 
breast  cancer  diagnosis  in  1993  did  not  have  a 
patient  encounter  with  the  reported  divisions 
at  Mayo  in  this  calendar  year.   When  including 
only  those  women  seen  at  the  sites  analyzed, 
94%  were  diagnosed  at  an  early  stage.   This 
rate  did  not  change  when  we  excluded  those 
aged  7  5  and  older.   The  percent  of  stage  2  and 
lymph  node  involvement  in  these  patients  were 
similarly  affected  by  a  small  percent  (Table 
2)  . 

DISCUSSION 

The  data  presented  here  reflects 
practice  prior  to  guideline  implementation. 
In  comparison  with  data  in  the  literature  it 
needs  to  be  stressed  that  our  rates  are 
encounter-based  and  not  population-based. 
Also,  the  data  includes  only  those  mammograms 
done  at  Mayo.  Similarly,  in  considering  the 
mammography  rates  for  those  women  with  general 
medical  or  pelvic  exams  the  preventive 
services  are  captured  only  if  the  exam  was 
performed  at  Mayo.  Any  mammogram,  general 
medical  or  pelvic  exams  done  outside  of  the 
Mayo  system  are  not  reflected  in   these 
percentages.   As  expected,  the  urgent  care 
center  is  affected  more  by  this  limitation  in 
the  data  than  some  of  the  other  divisions  due 
to  a  higher  likelihood  of  these  patients 
having  received  medical  care  at  a  facility 
other  than  the  Mayo  Clinic. 

To  assess  the  comparability  to 
population-based  rates,  mammography  screening 
rates  have  also  been  calculated  for  a  defined 
population  of  females  between  the  ages  of  50- 
74  who  are  enrolled  in  the  Mayo  Employee 
health  plan  during  the  1993  calendar  year. 
This  plan  includes  all  employees,  retirees, 
and  eligible  spouses  and  dependents  of  Mayo 
employees.   This  population  has  better  access 
and  coverage  than  the  general  population,  but 
may  be  similar  to  other  health  plans  in  the 
Midwest.   The  overall  baseline  screening  rate 
in  1993  for  this  population  was  60%  (figure 
1) .  The  comparison  of  rates  for  each  age  group 
with  those  for  all  patients  seen  is  also 
displayed.  The  screening  rates  for  the  defined 
population  was  slightly  lower  than  the 


encounter  based  rates  across  all  age  groups. 
Approximately  6%  of  the  women  overlap  between 
the  encounter-based  data  and  the  defined 
population  of  the  Mayo  Employee  Health  plan. 

To  date,  the  breast  disease  guidelines 
have  been  implemented  in  four  of  the  primary 
care  divisions.   The  rural  family  practice 
located  in  Kasson  began  implementation  in 
October  of  1994.   The  divisions  of  Area 
Medicine,  Community  Internal  Medicine  and 
Rochester  Family  Practice  implemented  these 
guidelines  during  1st  quarter  of  1995.   There 
has  been  a  mixed  reaction  to  the  guidelines  by 
physicians  within  the  primary  care  divisions. 
The  initial  reaction  is  to  assume  that  the 
current  practice  is  already  following  the 
guideline  recommendations.   After  reviewing 
the  baseline  numbers  specific  to  their 
division  they  are  much  more  receptive  to  the 
guideline  and 

implementation  efforts  taking  place.   To  date, 
the  four  groups  are  focusing  on  different 
aspects  of  reducing  barriers  and 
inefficiencies  in  their  current  process. 
Potential  access  limitations  with  efforts  to 
increase  the  patient  demand  for  services  have 
also  been  raised.  It  is  still  too  early  to 
assess  the  results  of  the  implementation 
efforts,  since  this  data  is  not  yet  available. 

In  conclusion,  there  are  four  main 
issues  raised  in  our  study: 

1.  Measuring  compliance  with  guidelines 
needs  to  address  issues  within 
control  of  the  implementation  team 
and  must  be  clinically  relevant. 
Measures  developed  by  Health  plans 
(i.e.  HEDIS)  may  need  modification 
for  use  by  a  f ee-f or-service  clinic. 

2.  Variability  of  performance  suggests 
that  room  for  improvement  exists  even 
among  high  quality  providers. 

3.  Practitioner  group  specific 
mammography  rates  provide  a  useful 
basis  for  the  identification  of 
specific  barriers  and  lead  to  further 
understanding  of  issues  in  the 
delivery  of  health  care. 

4.  The  appropriateness  of  mammography 
screening  among  40-49  year  old 
females  is  still  unsettled. 
Guidelines  and  their  measurements 
need  to  be  continuously  adapted  to 
changes  in  the  "best  practice"  of 
medicine . 
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Summary 

Breast  cancer  takes  the  highest  mortality 
rate  among  women  aged  35-64  years  in  the  European 
Community.  In  Germany,  the  mortality  rates  of 
female  breast  cancer  could  not  be  reduced  during 
the  eighties.  Detailed  regional  analyses  were  not 
carried  out  systematically.  An  overall  cancer 
register  does  not  exist. 

In  North  Rhine-Westphalia,  with  17  million 
inhabitants  the  largest  of  the  16  states  in 
Germany,  the  mortality  rates  of  female  breast 
cancer  in  the  age  group  from  35  -  64  years 
amounted  to  46.2  per  100,000  females  in  1980  and 
remained  unchanged  up  to  1993  with  a  mortality 
rate  of  46.5.  The  data  are  standardized  to  the 
„new"  European  population. 

An  analysis  of  the  54  counties  of  North 
Rhine-Westphalia  shows  differences  among  the 
regions,  but  only  one  region,  the  county  of 
Aachen,  shows  significantly  deviating  rates  below 
the  average.  Another  analysis  shows  the  trend  in 
breast  cancer  mortality  rates  by  5-year  age 
groups  &  15  years. 

The  second  of  the  proposed  10  health 
objectives  for  North  Rhine-Westphalia  relates  to 
cancer  control.  Indicators  like  mortality  rates 
(among  them  avoidable  cancer  mortality  up  to  65 
years  of  age),  incidence  rates,  (among  them  age- 
specific  incidence  rates  of  female  breast 
cancer),  coverage  rates  for  early  cancer 
detection  programs  and  additional  indicators  of 
the  effectiveness  of  early  detection  programs  and 
treatment  (inpatient  morbidity,  inability  to 
work)  are  analysed. 

Introduction 

In  the  latest  edition  of  the  "Europa-Blatter 
about  Health"  [Euroblatter  Nr.  2/95,  March/April 
(14)]  an  analysis  of  the  mortality  statistics  on 
the  level  of  the  European  Commity  and  World 
Health  Organization  was  published.  The  dominant 
cause  of  death  for  women,  aged  35-39  years  and 
40-64  years  was  breast  cancer.  We  all  must  be 
worried  about  this  and  analyse  the  situation. 

Latest  literature  about  trends  in  incidence 
and  mortality  of  female  breast  cancer  does  not 
give  important  new  information.  I  will  try  to 
give  a  short  survey  about  the  situation  in  diffe- 
rent European  countries  and  in  the  USA  and 
compare  these  data  with  German  data  -  analysed 
for  North  Rhine-Westphalia,  with  17  million  inha- 
bitants the  largest  of  the  16  German  states. 

In  Sweden  the  reduction  of  mortality  of 
breast  cancer  is  put  down  to  a  permanent  risk 
reduction  associated  with  screening  in  all  ana- 
lysed 5  studies  carried  out  with  282,777  women 
who  were  followed  up  in  randomised  trials  over  a 
period  of  5-13  years. 

The  largest  reduction  of  breast  cancer 
mortality  (29  %)  was  observed  among  women  aged  50 
-  69  years  at  randomisation.  Among  40  -  49-year- 
old  women  there  was  a  non-significant  13  %  reduc- 
tion [Nystroem,  L.  et  al.,  (1)]  Among  70  -  74 
year-old-women  screening  seems  to  have  had  only  a 
marginal  impact. 

The  National  Center  for  Health  Statistics 
(USA)  analysed  temporal  trends  in  breast  cancer 
mortality  among  US  women  who  were  examined  from 


1969  through  to  1989  by  age,  race  and  county- 
level  socioeconomic  status.  The  mortality  ratio 
in  counties  with  a  high  -  relative  -  to  low  so- 
cioeconomic status  declined  significantly  among 
women.  A  relative  increase  in  breast  cancer 
incidence  could  be  registered  in  counties  with  a 
lower  socioeconomic  status.  [Wagener,  DK.; 
Schatzkin,  A  (4)]. 

The  National  Cancer  Institute  [Glass,  AG.; 
Hoover  RN.,(2)]  analysed  breast  cancer  incidence 
from  1960  to  1985.  The  incidence  rate  rose  by 
45  %  during  this  period.  The  largest  increases 
occured  among  60-year-old  or  older  women,  (74  %) 
and  among  45  -  69-year-old-women  (36  %). 

Sondik  (NCHS,  USA)  described  a  steady 
increase  in  the  rate  of  breast  cancer  cases  since 
1950,  with  a  sharp  rise  in  the  1980s  because  of 
the  increased  use  of  mammography.  Conversely, 
mortality  since  1973  had  declined  by  11  %  for 
women  younger  than  50  (the  decrease  occurred 
primarily  in  the  70s)  and  increased  by  6  %  for 
women  aged  50  and  older  [Sondik,  EJ.,  (5)]. 

In  1994  Kopans  (USA)  [Kopans ,  DB,  (6)] 
published  in  the  Journal  of  the  American  Cancer 
Society,  that  in  5  of  the  analysed  8  randomized 
trials,  controlled  trials  have  demonstrated  a 
mortality  reduction  for  screened  women  that 
ranges  between  22  %  and  49  %.  He  concluded,  that 
the  recent  withdrawal  of  screening  support  for 
40-  to  4  9-year-old-women  is  not  scientifically 
supported.  But  other  studies  show  that  there  is 
no  difference  in  survival  for  40-49-year-old 
women  and/or  for  50-59-year-old  women. 

This  corresponds  to  findings  by  Garfinkel, 
Boring,  Heath  [Garfinkel,  L.  et  al.(7)],  that  the 
increase  in  breast  cancer  incidence  coincides 
with  an  increased  use  of  mammography  in  asympto- 
matic women  in  the  1980s.  Mortality  from  breast 
cancer  has  changed  little  since  the  1930s,  but 
the  increases  in  localized  and  small-size  tumors 
and  decreases  in  the  rate  of  tumors  of  3  cm  or 
larger  at  diagnosis  indicate  that  breast  cancer 
mortality  may  start  to  decrease. 

After  analysing  a  group  of  women  enrolled  in 
the  Mamma  Breast  Self-Examination  Program, 
Gastrin  et  al .  (Finland)  [Gastrin,  G.,  (8)]  con- 
cluded that  the  reduction  in  mortality  from 
breast  cancer  in  the  study  group  is  consistent 
with  an  effect  of  the  Breast-Self-Examination- 
containing  Mamma  Program. 

For  the  former  German  Democratic  Republic 
Ebeling  reported  an  annual  mamma  carcinoma  inci- 
dence rate  of  60  per  100,000  women  [Ebeling,  K., 
(3)].  Latest  figures  provided  by  the  cancer 
registry  for  the  new  federal  states  [(M.  Mohner  et 
al.  (12)]  (former  GDR)  assume  a  rough  incidence 
rate  of  72  diseases  per  100,000  women  by  the  year 
1989.  In  1973  already,  Bardehle  [D.  Klessen 
(Bardehle,  13)]  calculated  a  rough  incidence  rate 
of  72  diseases  per  100,000  women  for  East  Berlin, 
a  rate  which  at  that  time  was  by  30  %  higher  than 
the  average  rate  for  the  GDR. 

As  to  the  old  Federal  States  the  incidence 
rate,  based  on  the  cancer  registry  of  the  Saar- 
land corresponds  to  87.5  diseases  per  100,000 
women  with  a  mortality  rate  of  37.9  per  100,000 
women  (age  standardized  rates  for  the  population 
of  the  Federal  Republic  of  Germany  of  the  year 
1987)  [B.  Pesch  et  al.;  Krebsatlas  Nordrhein-West- 
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falen  (11,)].  The  incidence  rates  of  the  cancer 
registry  Miinster  for  a  part  of  the  State  North- 
rhine-Westphalia  in  1989  amounted  to  60  and  rose 
to  70  in  1992. 
(Standardization  for  old  European  population). 

Breast  Cancer  Mortality  in  North  Rhine-Westphalia 

(NRW) 

Publications  about  Years  of  Potential  Life  Lost 
(YPLL)  in  the  United  States  and  calculations  for 
North  Rhine-Westphalia  with  the  same  method  show 
an  elevated  level  for  all  causes  of  death  in  the 
United  States,  but  in  the  field  of  cancer  the 
years  of  potential  life  lost  are  higher  in  North 
Rhine-Westphalia.  Therefore  it  would  be  important 
to  analyse  mortality  rates  of  female  breast  can- 
cer for  Germany  and  for  the  United  States  of 
America  (table  1). 

For  comparing  breast  cancer  mortality 
between  the  USA  and  North  Rhine-Westphalia  over 
the  years  1980  to  1993  the  „new"  European 
standard  population  was  used,  a  population 
created  by  the  UN  on  the  basis  of  the  population 
cross-section  of  all  European  states  of  the  year 
1991  (table  2).  Since,  compared  with  this 
standard  population,  the  female  German  population 
is  older,  many  of  the  standardized  figures,  which 
are  related  to  diseases  occuring  more  frequently 
at  a  higher  age,  will  be  lower  than  the  socalled 
„rough"  non-standardized  rates.  Standardization 
of  mortality  rates  due  to  breast  cancer  in  the 
United  States  led  to  following  results: 

The  rates  for  NRW  which  includes  all  age 
groups  shows  following  deviations  (figure  1): 

In  North  Rhine-Westphalia,  the  mortality 
rates  due  to  female  breast  cancer,  related  to  all 
age  groups  and  standardized  with  regard  to  the 
„new"  European  population  rose  from  38.6  in  1980 
to  40.4  in  1993.  This  corresponds  to  a  slightly 
positive  linear  trend  with  the  data: 

F(t)    =  37,79  +  0,21  (t)  corr.  coeff.  r  =  0,70 
standard  deviation  1,24 

Trend  1980  -  1993  by  Age  Groups,  NRW 

As  to  North  Rhine-Westphalia,  breast  cancer 
mortality  was  analysed  with  respect  to  various 
criteria,  e.g.  for  35-64  year-old  women,  for  25- 
64-year-old  women,  for  women  according  to  5-year- 
age  groups  and  for  all  women  within  temporal 
trends . 

For  the  35-64-year-old  women  mortality  rates 
for  the  years  1980  to  1993  were  calculated  for 
all  counties  and  the  State  North  Rhine-West- 
phalia. A  comparison  of  age  standardized  rates 
shows  that,  apart  from  minor  deviations,  the 
mortality  rate  of  46.2  per  100,000  of  the  35-64 
year-old  women  from  the  year  1980  did  not  change 
up  to  1993,  in  which  the  mortality  rate  amounted 
to  46.5  per  100,000  of  the  35-64  year-old  women. 
An  assessment  of  these  figures  which  remained 
about  unchanged  requires  a  comparison  with  the 
incidence  rates  which  are  available  from  1987  up 
to  1992  for  a  part  of  the  State  North  Rhine- 
Westphalia.  In  this  context  of  increasing  inci- 
dence rates  and  unchanged  mortality  rates  due  to 
breast  cancer  the  unchanged  trend  of  mortality 
rates  alltogether  does  not  represent  an  unfavou- 
rable development  (figure  2). 

For  a  more  detailed  evaluation  of  breast 
cancer  for  the  years  1980  -  1993  linear  trend 
functions  for  the  5-year-age  groups  from  25-29 
years  up  to  85  years  and  older  were  calculated 
and  shown  in  a  diagram  (figure  3)  .  The  following 
is  noticeable:  In  each  of  the  following  5-year- 
age   groups   mortality   is   higher   than   in   the 


preceeding  one.  Hence,  the  25-29-year-old  repre- 
sent the  lowest  mortality  rate,  the  85-year  old 
and  older  women  the  highest.  The  correlation 
coefficient  for  the  lowest  age  groups  is  nega- 
tive, while  with  increasing  age  the  positive 
value  of  the  correlation  coefficient  rises.  The 
increase  in  mortality  starts  above  all  with  the 
65-year-old  and  older  women.  The  diagramm  becomes 
clear  when  for  the  average  figures  of  the  years 
1980  to  1982  the  value  „100"  is  chosen  for  all 
age  groups  and  the  mortality  trend  of  the  average 
death  rates  up  to  the  years  1991  -  1993  is  shown 
(figure  4).  It  is  clearly  recognizable  that  the 
mortality  rate  of  the  younger  age  groups  could  be 
reduced,  but  that  on  the  other  hand  the  mortality 
rate  of  the  65-year-old  and  older  women  rose  by 
10  to  20  %.  Hence  the  unchanged  mortality  rate 
for  all  age  groups  conceals  a  considerable 
differentiation  when  analysing  mortality  by  5- 
year-age  groups. 

Regional  Differences 

North  Rhine-Westphalia  is  subdivided  into  54 
counties  and  independent  municipalities  with  a 
population  of  at  least  200,000  inhabitants  on 
average.  For  a  comparison  by  regions  for  the 
years  1980  -  1993  mortality,  due  to  the  low  fi- 
gures, was  not  only  calculated  for  all  counties 
according  to  individual  years,  but  in  addition  an 
age  standardization  was  carried  out  for  all  14 
years  that  were  taken  as  a  basis  (in  accordance 
with  the  „new"  European  population)  .  The  35-64- 
year-old  women  represent  the  so-called  „truncated 
population",  which  in  North  Rhine-Westphalia  over 
the  14  years  amounts  to  an  average  mortality  rate 
of  46.7  per  100,000  women  of  the  respective  age 
group  (standardized  to  the  „new"  European  popu- 
lation) (figure  5).  The  highest  average  figure  is 
shown  by  the  county  of  Heme  with  a  mortality 
rate  of  54.8  per  100,000  of  the  35-64-year-old 
women  (age  standardized),  the  lowest  figure  by 
the  county  of  Paderborn  with  a  mortality  rate  of 
38.6  per  100,000  of  the  35-64-year-old  women  (age 
standardized) . 

Avoidable  Mortality 

According  to  the  proposals  by  a  working 
group  of  the  European  Union  "avoidable  cases  of 
death"  include  those  which  are  meant  to  stand  for 
effectiveness  and  quality  of  health  care,  e.g. 
within  the  framework  of  early  recognition  pro- 
grams or  successful  early  treatment.  On  the  other 
hand  those  cases  of  death  are  excluded  which  are 
caused  by  negligence  or  culpable  failure  of  indi- 
vidual persons.  At  present  mortality  is  restric- 
ted to  12  causes  which  showed  significant  diffe- 
rences among  the  regions  when  an  analysis  was 
carried  out  in  one  state  of  the  Federal  Republic 
of  Germany  (Rhineland-Palatinate )  .  This  includes 
female  breast  cancer  of  women,  aged  25-64  years. 

On  this  methodical  basis  the  Standardized 
Mortality  Ratio  (SMR)  for  the  years  1989-1993  was 
calculated  for  the  25-64-year-old  women  who  died 
as  a  consequence  of  breast  cancer  within  these 
five  years  (figure  6).  In  addition  the  figure  was 
checked  with  the  confidence  interval.  The  re- 
striction to  this  age  group  was  made  because  for 
the  under  65-year-old  women  epidemiological  chan- 
ges above  all  in  relation  to  the  reduction  of 
mortality  rates  became  obvious.  The  SMR  was 
related  to  the  mean  of  the  State  North  Rhine- 
Westphalia  for  the  observation  period  1989 
1993.  Accordingly,  the  SMR  takes  the  value  1.0  if 
the  Specific  mortality  rate  in  one  county  exactly 
corresponds  to  the  state  average.  Within  the  54 
counties  the  SMR  ranges  between  0.67  and  1.21.  In 
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this  connection  no  county  shows  a  significantly 
elevated  sMR,  two  counties  are  significantly 
below  the  state  average  (significance  calculated 
using  95  per  cent  confidence  intervals). 

The  Epidemiological   Situation  of  Female  Breast 

Cancer 

Incidence  of   the  Mamma  Carcinoma   1987   -   1992, 

Cancer  Registry  Munster 

For  10  years  the  cancer  registry  in  Munster 
has  been  registering  the  incidences  of  all  mali- 
gnant tumors  for  a  residential  area  of  2.3 
million  inhabitants  of  the  State  North  Rhine- 
Westphalia.  The  registrations  are  80-85  %  com- 
plete. The  incidence  rates,  standardized  to  the 
„old"  European  population,  show  an  increase  from 
60  to  70  per  100,000  of  the  female  population 
from  the  years  1987  to  1992  (figure  7). 

Trend  observations  of  incidence  as  well  as 
mortality  rates  by  5-year  age  groups  lead  to 
following  conclusions  for  the  years  1987  to  1992: 

Above  all  among  the  40  to  64-year-old  women 
the  incidence  rates  rose  by  following  values: 
related  to  100,000  women  of  the  respective  age 
group: 


age  group 


1987 


1992 


40-44 

years 

55 

.3 

83 

.  5 

45-49 

years 

92 

.0 

130 

.4 

50-54 

years 

115 

.3 

160 

.9 

55-59 

years 

130 

.1 

160 

.4 

60-64 

years 

143 

,4 

168 

.4 

Over  this  period  from  1987  to  1992  mortality 
rates  per  100,000  women  for  the  same  age  groups 
developed  as  follows: 


age  group 


1987 


1992 


40-44 

years 

23 

.4 

30 

.7 

45-49 

years 

45 

.1 

36 

.8 

50-54 

years 

56 

.3 

57 

.1 

55-59 

years 

74 

.1 

77 

.4 

60-64 

years 

81 

.1 

85 

,3 

Interpretation:  The  quotient  of  the  women 
who  fell  ill  in  1987  and  died  on  average  was  2:1 
in  favour  of  the  incidences.  In  1992  a  higher 
incidence  rate  was  registered  for  all  age  groups 
and  a  ratio  between  incidence  and  mortality  which 
for  the  younger  age  groups  amounted  to  3:1  in 
favour  of  the  incidences.  As  to  the  50-54-year- 
old  women,  the  absolute  figures  of  the  women 
suffering  from  breast  cancer  within  the  county  of 
Munster  in  1987  amounted  to  91,  in  1992  to  137 
women.  It  cannot  be  excluded,  that  variations  are 
due  to  low  figures  and  that  the  registration 
quota  may  be  different.  The  trend  of  increasing 
incidence  rates  with  relatively  unchanged  morta- 
lity rates  as  described  in  literature  is  however 
comprehensible . 

For  characterizing  the  epidemiological 
situation,  incidence  rates  (source:  Federal  AOK 
Association),  the  rate  of  patients  treated  in 
hospitals  (source:  ditto),  the  rate  of  patients 
unable  to  work  (source:  ditto)  and  mortality 
rates  (source:  State  Office  for  Data  Processing 
and  Statistics  for  the  State  North  Rhine- 
Westphalia)  are  shown  (figure  8).  It  is  clearly 
recognizable  that  all  four  indicators  mentioned 
show  one  break  at  the  age  between  45  and  69 
years,  hence  underline  the  double  peak  character 
of  the  disease.  The  rate  of  women  treated  in 
hospitals  represents  the  highest  figures,  at  old 
age  treatment  rates  decrease  -  as  we  all  know  - 
incidence  rates  increase  and  mortality  rates 
strongly   increase.   Inability   to  work   abruptly 


from  the  age  of  20  up: 
from  the  age  of  30  up: 
from  the  age  of  45  up: 


stops  as  soon  as  these  women  enter  their 
retirement  age.  This  is  how  the  situation  in 
terms  of  epidemiology  and  treatment  in  North 
Rhine-Westphalia  can  be  described. 

Early  Detection  of  Cancer  -  Programs  and  Results 
in  NRW 

In  1971  the  statutory  program  for  early 
detection  of  cancer  was  introduced  in  the  Federal 
Republic  of  Germany  and  has  been  modified  several 
times.  [Krankheitsf ruherkennung  Krebs,  (9)]. 

At  present  it  includes  following  examina- 
tions for  women: 

examination  for  cancer  of 
the  genitals 

in   addition   examinations 
for  breast  and  skin  cancer 
in   addition   examinations 
for  cancer  of  the  rectum 
and  colon 
The  program  for  early  detection  of  breast 
cancer  includes  breast  palpation  and  instructing 
women  on  self-examination  of  the  breast.  Mammo- 
graphy is  considered  a  method  of  diagnostic  cla- 
rification and  is  not  part  of  the  program  for 
early   detection   of   cancer.   According   to   our 
information  about  3.3  million  women  of  the  State 
NRW  annually  take  part  in  the  examinations,  rela- 
ted to  all  types  of  cancer  mentioned  in  the 
program. 

The  age  standardized  discovery  rates  of 
breast  cancer  varied  considerably  between  the  two 
regions  of  Westphalia-Lippe  and  North  Rhine  and 
at  first  examinations  in  Westphalia-Lippe  amoun- 
ted to  about  100  discovered  mamma  carcinomas  per 
100,000  women  examined  for  the  first  time  and  to 
about  55  discovered  mamma  carcinomas  per  100,000 
examined  women  (first  and  repeated  examinations 
at  different  intervals).  In  North  Rhine,  with 
about  20  -  25  discovered  mamma  carcinomas  per 
100,000  of  the  women  examined  for  the  first  time 
and  about  15  mamma  carcinomas  per  100,000  of  the 
repeatedly  examined  women  (first  and  repeated 
examinations),  these  rates  differed  considerably 
and  were  below  the  average  of  the  whole  Federal 
Republic  of  Germany.  The  rate  of  the  non- 
confirmed  diagnoses  amounted  to  about  90  %,  the 
confirmed  diagnoses  to  about  3  %,  the  remaining 
7  %  cannot  be  determined.  The  hence  calculated 
discovery  rate  for  the  Federal  Republic  of 
Germany  in  1990  amounted  to  41  breast  cancer 
incidences  per  100,000  examined  women  aged  30  or 
older,  whereas  in  1989  96  mamma  carcinomas  per 
100,000  women  aged  30  or  older  were  discovered. 
These  varia-tions  between  the  regions  in  North 
Rhine-Westphalia  and  between  the  results  of  the 
years  1989  and  1990  show  how  difficult  an 
evaluation  of  data  for  a  whole  state  is,  data 
which  are  based  on  a  participation  rate  of 
annually  over  30  %  of  the  entitled  female  popu- 
lation [Bernt-Peter  Robra,  Evaluation  des  deut- 
schen  Krebsf ruherkennungsprogramms ,  (10)] 

Conclusions : 

The  G7  summit  in  Brussels  in  February  1995 
with  the  member  states  Canada,  France,  Germany, 
Italy,  Japan,  United  Kingdom,  USA,  and  with  the 
European  Commission  and  WHO  set  up  a  global 
telematics  programme  to  facilitate  access  to  the 
relevant  public  health  knowledge  bases  in  the  G7 
member  states:  to  be  prepared  by  the  end  of  1996. 
Two  sectors  have  been  selected  for  testing: 
communicable  diseases  and  vital  statistics  to  be 
extended  to  health  indicators  next.  To  prove  the 
validity  of  one's  own  data  like  in  the  field  of 
breast  cancer,   availability  of  data  and  their 
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comparison  with  US  data  will  be  one  step  in  the 
direction  of  a  G7  telematics  information  system. 

The  problem  of  high  rates  of  breast  cancer 
mortality  could  be  an  important  health  indicator 
for  evaluating  the  effectiveness  of  early 
detection  programs.  The  exchange  of  data  with  the 
help  of  telecommunication  technology  and  provi- 
ding the  means  for  a  comparability  of  data  among 
the  mentioned  7  states  could  be  a  very  attractive 
task  for  all  of  us. 
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Table  1: 


Years  of  Potential  Life  Lost,  YPLL 

Years  lost  per  100  000  population  under  65  years 
of  age 

Causes  of  Death 

USA,  1991 

NRW,  1993 

All  causes 

5556       100% 

3463        1 00  % 

Cancer 

843       15,2% 

1 023         29,5  % 

ICD  140-208 

Heart  disease  and 
stroke 

737       13,3% 

656         19,0% 

ICD  390  -  436 
(401,403excl.) 

Intentional  and 
unintent.  injuries 

1637      29,5% 

600         17,3% 

ICD  800  -  999 

HIV /AIDS 

347       6,3  % 

69          2,0  % 

ICD  42 -44 

Source:  CDC,  National  Center  for  health  Statistics, 
U.S.A. 

State  Institute  for  Public  Health,  North  Rhine- 
Westphalia 


Table    2: 
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"Old"  and  "New"  European  Standard  Population  by 
age  groups  and  sex 

age 
groups 

"Old" 
European 
Population 

1966 

"New"  European  Population  1991 

total          of  which        of  which 
male            female 

0-1 

1.600 

1.305 

1.345 

1.218 

1  -4 

6.400 

5.021 

5.303 

4.800 

5-9 

7.000 

6.472 

6.800 

6.160 

10-14 

7.000 

6.772 

7.108 

6.452 

15-19 

7.000 

7.208 

7.570 

6.863 

20-24 

7.000 

7.792 

8.163 

7.438 

25-29 

7.000 

7.871 

8.206 

7.552 

30-34 

7.000 

7.528 

7.811 

7.258 

35-39 

7.000 

7.212 

7.448 

6.986 

40-44 

7.000 

6.860 

7.168 

6.661 

45-49 

7.000 

5.865 

5.997 

5.739 

50-54 

7.000 

5.876 

5.937 

5.817 

55-59 

6.000 

5.553 

5.521 

5.585 

60-64 

5.000 

5.245 

5.015 

5.463 

65-69 

4.000 

4.680 

4.139 

5.196 

70-74 

3.000 

2.932 

2.449 

3.392 

75-79 

2  000 

2.897 

2.228 

3.536 

80-84 

1.000 

1.606 

1.094 

2.076 

85  + 

1.000 

1.305 

798 

1.808 

Total 

100.000 

100.000 

100.000 

100.000 
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Figure  1: 


Breast  cancer  mortality  per  100  000  women  ',  North  Rhine-Westphalia,  1980  - 

1993 
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Figure  3: 


Mortality  rates  of  breast  cancer  (ICO  174),  North  Rhlne-Westphalla,  1980  -  1993 


Cases  of  death  per  100  000  by  the  respective  age  groups,  femalo 
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Figure   2  : 


Incidence-  and  mortality  rate  of  breast  cancer,  50  -  54-year-old  women,  North 
Rhine-Westphalia,  1987  - 1992 


per  100  000  of  the  respective  age  group,  female 
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Figure   4: 


Mortality  rates  of  breast  cancer  (ICD  174),  North  Rhine-Westphalia,  Mean  1980  - 
1982  and  1991  -1993 


Deviation  in  %  from  1980  -  1982  to  1991  -  1993 
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Figure   5: 


Figure   6: 


Mortality  rate  of  breast  cancer, 

35  -  64  year-old  women,  North  0^ 

Rhine-Westphalia,  Mean  1991  - 1993 


Figure   7; 


Figure   8: 


Incidence-  and  mortality  rate  of  breast  cancer,  selected  age  groups,  North 
Rhine-Westphalia,  1987  - 1992 


per  100  000  of  the  respective  age  groups,  female 
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AT  RISK  IN  AMERICA:   HEALTH  AND  HEALTH  CARE 
FOR  VULNERABLE  POPULATIONS 

Lu  Ann  Aday,  The  University  of  Texas  School  of  Public  Health 


WHO  ARE  THE  VULNERABLE? 

Vulnerable  individuals  are 
"susceptible  to  harm  or  neglect". 
The  word  "vulnerable"  is  derived  from 
the  Latin  verb  "vulnerare"  ("to 
wound")  and  the  noun  "vulnus" 
("wound") (Aday,  1993). 

"Vulnerable  populations  are  at 
risk  of  poor  physical,  psychological, 
or  social  health." 

Health  can  be  measured  along  a 
continuum  of  seriousness  with  good 
health  being  at  the  positive  end  of 
the  continuum,  defined  by  the  World 
Health  Organization  (WHO)  concept  of 
health  as  a  "state  of  complete 
physical .  mental .  and  social  well- 
being..."  (World  Health  Organization, 
1948,  p.  1),  and  death  at  the 
negative  end,  as  the  total  absence  of 
health,  defined  by  population- 
specific  mortality  (death)  rates. 

Underlying  this  definition  of 
vulnerability  is  the  epidemiological 
concept  of  risk,  in  the  sense  that 
there  is  a  probability  that  an 
individual  will  become  ill  within  a 
stated  period  of  time.   Given  this 
sense  of  the  notion  of  risk,  we  are 
all  vulnerable,  that  is,  there  is 
always  a  chance  (or  a  non-zero 
probability)  that  we  will  become  ill 
at  some  time  in  our  lives. 

Vulnerable  populations  may  be 
identified  based  on  those  for  whom 
the  risk  of  poor  physical, 
psychological  or  social  health  has  or 
is  quite  likely  to  become  a  reality: 

physical:   high-risk  mothers  and 
infants,  chronically  ill  and 
disabled,  persons  with  AIDs; 

mental:   mentally  ill  and 
disabled,  alcohol  or  substance 
abusers,  suicide-  or  homicide-prone; 
and 

social:   abusing  families,  the 
homeless,  and  immigrants  and 
refugees. 

There  is,  of  course,  overlap 
among  these  groups,  and  the 
boundaries  should  be  viewed  to  be 
diffuse  rather  than  distinct. 

Poor  health  along  one  dimension 
(physical)  is  quite  likely  to  be 
compounded  with  poor  health  along 
others  (psychological  and/or  social, 
for  example) .   Health  needs  are 
greatest  for  those  who  have  problems 
along  more  than  one  of  these 
dimensions. 

WHY  ARE  THEY  VULNERABLE? 

Two  different  mother  tongues  — 


those  of  "individual  rights"  and  the 
"common  good"  —  have  historically 
characterized  American  social  and 
political  discourse.   The  semantics 
of  the  first  emphasize  the  meanings 
of  autonomy,  independence,  and 
individual  well-being,  while  the 
second  highlights  norms  of 
reciprocity,  interdependence,  and  the 
public  good.   Robert  Bellah,  et  al. 
(1985)  and  others  (Beauchamp,  1988; 
Tesh,  1988)  have,  however,  observed 
that  in  contemporary  American 
society,  the  first  language  of 
individualism  has  come  to  override 
the  second  mother  tongue  of 
community.   Excessive  individualism 
may,  he  argues,  be  "destroying  those 
social  integuments  [ties]  that  de 
Tocqueville  saw  as  moderating  its 
more  destructive  potentialities" 
(Bellah,  et  al.,  1985:   viii) . 

James  Coleman  (1990) ,  in  his 
book,  Foundations  of  Social  Theory, 
points  out  that  to  formulate 
meaningful  theories  or  explanations 
of  social  phenomenon,  both  the  macro 
(collective)  as  well  as  micro 
(individual)  levels  of  observation 
and  analysis  and  their 
interrelationships  must  be  examined. 
Focusing  on  individuals' 
characteristics,  attitudes  or 
behaviors  (violence-proneness)  may 
fail  to  reveal  the  impact  that  larger 
social  influences  or  trends  (media 
violence  and  associated  societal 
tolerances  of  violence  toward  women 
and  children)  have  upon  the 
individuals  themselves. 
Correspondingly,  theories  regarding 
relationships  between  largely 
collective  phenomenon  (the  prevalence 
of  media  violence  and  rates  of 
violent  crime)  that  fail  to 
illuminate  the  dynamics  of  these 
social  forces  for  individuals  fall 
short  of  developing  fully  meaningful 
explanations  of  the  phenomenon.   The 
measurement  of  collective  phenomena 
at  the  individual  level  of  analysis 
(methodological  individualism)  also 
tends  to  bias  the  explanations  of 
these  phenomena  toward  individual 
motivations  and  actions. 

Ethical  Norms  and  Values 

An  individual  perspective  on  the 
origins  of  poor  health  views  personal 
autonomy,  independence  and  associated 
individual  rights  as  the  principal 
ethical  norms  and  values  for  guiding 
decision-making  regarding  the 
amelioration  of  risk.   Good  health  is 
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viewed  to  be  primarily  a  function  of 
personal  lifestyle  choices,  and  poor 
health  outcomes  result  because 
individuals  fail  to  assume  adequate 
personal  responsibility  for  their 
health  and  well-being. 

A  community  perspective  on  the 
origins  of  health  needs  focuses  on 
the  differential  risks  that  exist  for 
different  groups  as  a  function  of  the 
availability  of  opportunities  and 
resources  for  maximizing  their 
health.   Norms  of  reciprocity,  trust 
and  social  obligation  acknowledge  the 
webs  of  interdependence  and  mutual 
support  and  caring  that  are  essential 
for  minimizing  the  risks  of  poor 
health.   Poor  health  results  because 
communities  fail  to  invest  in  and 
assume  responsibility  for  the 
collective  well-being  of  their 
members. 

Needs  Assessments 

Individual  health  needs 
assessments  measure  the  health  status 
of  identifiable  community  residents 
or  patients  (based  on  symptoms  or 
diagnoses  of  illness,  for  example) . 

Community  health  needs 
assessments  focus  on  statistical 
indicators  of  the  rates  of  prevalence 
or  incidence  of  morbidity  or 
mortality  (such  as  infant  mortality 
rates,  HIV-seroprevalence,  percent  of 
the  elderly  with  limitations  in 
activities  of  daily  living,  and  so 
on)  . 

The  former  has  principally  been 
the  focus  of  personal  medical  care 
service  delivery  and  practice  and  the 
latter  of  public  health  policy  and 
planning. 

Relative  Risk 

Relative  risk  refers  to  the 
differential  vulnerability  of 
different  groups  to  poor  health. 
People  may  be  more  or  less  at-risk  of 
poor  health  at  different  times  in 
their  lives,  while  some  individuals 
and  groups  are  apt  to  be  more  at-risk 
than  others  at  any  given  point  in 
time. 

The  beginning  point  for 
understanding  the  factors  that 
increase  the  risk  of  poor  health 
originates  in  a  macro-level  look  at 
the  availability  and  distribution  of 
community  resources.   Individuals' 
risks  vary  as  a  function  of  the 
opportunities  and  material  and 
nonmaterial  resources  associated  with 
(1)  the  personal  characteristics 
(age,  sex,  and  race/ethnicity)  of  the 
individuals  themselves;  (2)  the 
nature  of  the  ties  between  them 
(family  members,  friends,  and 
neighbors,  for  example);  and  (3)  the 


schools,  jobs,  incomes  and  housing 
that  characterize  the  neighborhoods 
in  which  they  live. 

Social  status  is  associated  with 
positions  individuals  occupy  in 
society  as  a  function  of  age,  sex,  or 
race/ethnicity,  and  the  corollary 
socially  defined  opportunities  and 
rewards,  such  as  prestige  and  power, 
they  have  as  a  result. 

Joel  Kovel  (1984)  in  his  book, 
White  Racism:   A  Psvchohistory . 
distinguishes  dominative,  aversive, 
and  meta-racism.   Dominative  racism 
involves  enslavement  or  other  forms 
of  forced  labor,  which  characterized 
the  first  two  centuries  of  African- 
Americans'  experience  in  the  United 
States.   Aversive  racism  is  a  racism 
of  avoidance  and  separation, 
enforced,  for  example,  by  the 
"separate  but  equal"  provisions  and 
Jim  Crow  laws  that  continued  to 
direct  race  relations  into  the  first 
half  of  this  century.   Metaracism 
refers  to  a  muting  of  the  overt 
rhetoric  regarding  racial 
superiority,  and  the  sustained 
dominance  of  a  pervasive,  white- 
majority  control  of  the  economy, 
politics  and  cultural  standards  of 
truth  and  beauty,  that  characterize 
contemporary  U.S.  society. 

These  "-isms"  may  also  be 
applied  in  characterizing  the 
treatment  of  women  historically  and 
contemporarily:   as  largely  enslaved 
by  patriarchal  social  and  economic 
systems  of  dominance  to  the  exclusion 
of  women  from  the  right  to  vote  or 
the  workplace  to  the  pervasive  and 
insidious  forms  of  oppression  and 
marginalization  reflected  in  sexual 
stereotyping,  the  misogynist 
portrayal  of  women  in  the  media,  and 
the  social  and  economic  de-valuing  of 
the  roles  they  play  in  the  home  and 
the  marketplace. 

Social  capital  resides  in  the 
quantity  and  quality  of  interpersonal 
ties  between  people.   Families 
provide  social  capital  to  members  in 
the  form  of  social  networks  and 
support  and  associated  feelings  of 
belonging,  psychological  well-being 
and  self-esteem.   The  value  of  social 
capital  to  individuals  (single 
mothers)  is  that  it  provides 
resources  (such  as  having  someone  to 
count  on  for  child  care)  they  can  use 
to  achieve  other  interests  (going  to 
school  or  working) . 

Social  support  has  been  found  to 
be  an  important  resource  for 
individuals  in  coping  with  and 
minimizing  the  impact  of  negative 
life  events  or  adversity  on  their 
physical  and  mental  health. 
Physical,  psychological,  and  social 
well-being  are  directly  enhanced  for 
people  who  have  supportive  social 
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networks.   Communities  constitute  the 
reservoir  in  which  social  capital 
resources  are  both  generated  and 
drawn  upon  by  individual  community 
members.   Those  who  are  likely  to 
have  the  least  social  capital  (or  the 
fewest  social  ties  to  count  on)  are 
people  living  alone  or  those  in 
female-headed  families,  those  who  are 
not  married  or  in  an  otherwise 
committed  intimate  relationship, 
people  who  do  not  belong  to  any 
voluntary  organizations  (such  as 
churches  or  volunteer  interest 
groups) ,  or  have  weak  or  nonexistent 
social  networks  of  family  or  friends. 

Human  capital  refers  to 
investments  in  people's  skills  and 
capabilities  (such  as  vocational  or 
public  education)  that  enable  them  to 
act  in  new  ways  (master  a  trade)  or 
enhance  their  contributions  to 
society  (enter  the  labor  force) . 
Social  capital  can  also  enhance  the 
generation  of  human  capital  through, 
for  example,  family  and  community 
support  for  encouraging  students  to 
stay  in  school.   Neighborhoods  that 
have  poor  schools,  high  rates  of 
unemployment  and  substandard  housing 
reflect  low  levels  of  investments  in 
the  human  capital  (or  productive 
potential)  of  the  people  who  live 
there.   Similarly,  individuals  who 
are  poorly  educated,  unemployed  and 
poorly  housed  are  likely  to  have  the 
fewest  resources  for  coping  with 
illness  or  other  personal  or  economic 
adversities. 

WHO  IS  MOST  VULNERABLE? 

Those  individuals  with  a 
combination  of  statuses  (poor, 
elderly  women,  those  living  alone,  or 
minority  adolescents)  that  put  them 
at  a  high  risk  of  having  both  poor 
health  and  few  material  and 
nonmaterial  resources  are  in  a  highly 
vulnerable  position. 

The  1990s  have  been  and  promise 
to  continue  to  be  a  harmful,  not  a 
kind  and  gentle,  decade  for  many 
vulnerable  Americans. 

WHAT  DATA  PROBLEMS  EXIST? 

Ambiguity  of  Definitions 

A  fundamental  problem  in 
identifying  who  and  how  many  are 
vulnerable  is  ambiguity  in  the 
definitions  of  the  vulnerable 
populations  themselves. 

High-risk  mothers  and  infants 
have  been  identified  using  a  variety 
of  predictors  and  indicators  of  both 
morbidity  and  mortality  (such  as  low 
or  very  low  birthweight  infants  and 
maternal  and  infant  deaths) .   The 
chronically  ill  and  disabled  have 


been  defined  based  on  diagnoses, 
disability,  and  functional  status,  as 
well  as  guality  of  life  measures. 
The  case  definition  of  AIDS  has  been 
revised  three  times  since  it  was 
first  published  in  1982.   At  least 
four  generations  of  studies  in  the 
mental  health  field  can  be  identified 
in  this  century,  all  of  which  used 
either  different  approaches  or 
instruments  (record  sources  and  key 
informants,  psychiatric  clinical 
judgments,  standardized  surveys,  and 
measures  of  social  functioning) ,  for 
defining  cases  of  mental  illness. 

The  measurement  of  alcohol  or 
substance  abuse  is  made  more 
difficult  by  the  fact  that  there  are 
different  stages  in  the  development 
of  the  addictive  behaviors: 
nonaddictive  use,  excessive  use 
(abuse) ,  addictive  dependency,  and 
recovery  or  relapse.   Accurate 
reports  of  homicides  and  suicides 
depend  on  accurately  classifying  the 
intent  of  the  perpetrators  of  the 
acts  resulting  in  these  deaths. 
Maltreatment  can  include  both  acts  of 
commission  (abuse)  and  omission 
(neglect) ,  as  well  as  a  variety  of 
types  of  harm  or  endangerment 
(physical,  sexual,  or  emotional). 
The  condition  of  homelessness  can  be 
assessed  with  respect  to  time 
(temporarily,  episodically  or 
chronically  homeless)  or  location 
(living  on  the  streets,  in  temporary 
housing,  or  doubled  up  with 
relatives) .   Immigrants  and  refugees 
encompass  those  who  are  here  legally 
as  well  as  those  who  are  not,  and 
among  the  former,  settlers, 
sojourners  and  commuters  (permanent 
residents,  temporary  visitors,  and 
those  who  regularly  cross  the  Mexican 
or  Canadian  borders  to  work, 
respectively)  are  all  included. 

Quality  of  Data  Sources 

Another  important  problem  in 
estimating  the  number  (or  prevalence) 
and  health  status  of  people  who  are 
vulnerable  is  that  the  quality  and 
completeness  of  the  data  sources  for 
identifying  them  are  limited.   These 
sources  may  include  clinical 
diagnoses  of  disease,  patient  self- 
reports  of  illness,  vital  statistics 
inventories  on  births  and  deaths,  and 
health  and  social  service  agency 
records  on  clients. 

Different  sources  tend  to  yield 
variant  estimates  of  those  in  need 
within  a  particular  group,  which  also 
make  direct  comparisons  of  the 
magnitude  of  need  across  groups 
difficult.   Different  universes  (or 
groups)  of  individuals  are  used  as 
the  basis  for  different  estimates. 
Further,  estimates  based  on  survey 
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data  (such  as  the  prevalence  of 
alcohol  or  substance  abuse  or  family 
violence)  may  also  have  systematic 
biases  resulting  from  only  selected 
groups  or  individuals  being  included 
in  or  responding  to  the  survey,  as 
well  as  variable  (sampling)  errors 
associated  with  the  size  and 
complexity  of  the  sample  design. 

Changes  over  Time 

Trend  data  that  document 
increases  in  certain  types  of 
problems  (such  as  child  abuse  and 
neglect)  are  also  confounded  with  the 
increased  visibility  and  likelihood 
of  reporting  these  types  of  events. 
Often  data  are  not  available  in  a 
timely  fashion  or  vary  in  guality  and 
completeness  across  studies  and 
sources.   Cutbacks  in  funding,  as 
well  as  changes  in  definitions  of 
cases  over  time  (which  may  be 
warranted  to  capture  the  changing 
dynamics  of  the  problem,  such  as 
AIDS) ,  can  nonetheless  jeopardize  the 
availability  of  longitudinal  data  to 
trace  changes  in  the  incidence  or 
prevalence  of  these  problems. 

Lack  of  Subgroup  Data 

The  lack  of  demographic 
identifiers  or  detail  (by  race  and 
ethnicity,  for  example)  can  also 
limit  analyses  of  differences  between 
groups  for  whom  the  risk  or  magnitude 
of  problems  are  most  likely  to  vary. 

Many  people  have  more  than  one 
type  of  health  problem.   Low 
birthweight  babies  may  have 
congenital  defects  or  other  adverse 
outcomes  associated  with  prematurity 
that  result  in  long-term  physical  or 
mental  impairment.   Particularly 
high-risk  categories  of  mothers  and 
infants  include  those  in  which  the 
mother  or  her  sex  partner (s)  used 
drugs  or  were  HIV-positive.   Pregnant 
women  with  abusive  partners  or  those 
who  are  homeless  or  fleeing  political 
persecution  are  particularly  at  risk 
of  poor  outcomes  for  themselves  and 
their  unborn  children.   Accurate 
national  estimates  on  the  number  of 
these  and  other  groups  with  a 
multiplicity  of  cross-cutting  needs 
are  not  readily  available. 
Examination  of  data  for  discrete 
subgroups  should  not,  however, 
obscure  the  mosaic  of  physical, 
psychological,  and  social  needs  that 
characterize  the  lives  of  many  of  the 
vulnerable. 

WHAT  PROGRAMS  ARE  NEEDED? 

The  health  and  health  care  needs 
of  vulnerable  populations  are  best 
addressed  by  a  long-term,  prevention- 


oriented  continuum  of  programs  and 
services. 

The  programs  and  services  that 
might  comprise  such  a  continuum 
include  (1)  primary  prevention- 
oriented  community  resource 
development  and  public  health 
programs;  (2)  treatment-oriented  care 
delivered  primarily  through  the 
medical  care  and  related  professional 
service  delivery  systems;  and  (3) 
long-term  care  institutional  and 
community-based  programs  and 
services.   The  organization  and 
integration  (more  often  lack  of 
integration)  of  these  programs  does 
not  typically  acknowledge  the 
emergence  and  evolution  of 
vulnerability  over  the  life  course  of 
individuals,  as  well  as  its  essential 
roots  in  the  communities  from  which 
people  emerge,  and  to  which  they 
return  after  being  treated  by  the 
formal  professional  service  delivery 
systems. 

HOW  SHOULD  POLICIES  BE  RESHAPED? 

Heath  care  reform  is  proceeding 
at  a  rapid-fire  pace  at  the  state  and 
local  level,  through  the  widespread 
adoption  of  managed  care  in  both  the 
private  and  public  health  care 
sectors.   The  numbers  of  privately 
insured  individuals  enrolled  in  these 
arrangements  continue  to  grow,  as 
does  the  proliferation  of  federal 
waiver  requests  submitted  by  states 
to  expand  managed  care  under  the 
Medicaid  program. 

This  market-oriented  approach  to 
health  care  reform  focuses  on  the 
management  of  and  competition  between 
discrete  providers  of  services.   It 
is  manifest  in  the  proliferation  and 
consolidation  of  provider  networks 
into  integrated  systems  of  delivering 
and  financing  medical  care,  that 
impose  varying  constraints  on 
providers'  fees  and  consumers' 
utilization  of  services. 

Applying  community-oriented 
lenses  to  these  developments  would 
seek  to  illuminate  the  resultant 
distribution  of  and  linkages  between 
providers  along  a  continuum  of 
prevention-oriented,  treatment- 
oriented,  and  long-term  care. 

A  community-oriented  health 
policy  to  address  the  health  and 
health  care  needs  of  vulnerable 
populations  acknowledges  the 
essential  social  origins  and 
consequences  of  poor  physical, 
psychological  and/or  social 
functioning,  and  the  array  of 
community-based,  nonmedical  social 
and  community  support  services 
required  to  ameliorate  both  the  risk 
and  consequences  of  vulnerability. 
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It  also  considers  the 
distribution  of  programs  and  services 
across  social  and  economic  strata 
within  the  community. 

To  begin  to  attend  to  the 
dimensions  and  scope  of  the  problem 
of  vulnerability  to  poor  health  in 
the  United  States,  policymakers  must 
come  to  envision  a  sense  of 
community,  and  the  normative  compass 
and  context  it  provides. 

Features  of  such  a  policy  are, 
for  example,  manifest  in  the 
formation  of  community-based 
organizations  and  consortia  to 
develop  programs  and  services  for 
persons  with  AIDS,  the  homeless,  and 
chronically  mentally  ill;  the 
development  of  community-oriented 
primary  care  (COPC)  models  of  service 
delivery;  school-b^sed  clinics;  and 
client-  or  family-centered  outreach 
and  case  management  services. 

A  broader  set  of  goals  and 
objectives  is  required  to  more  fully 
capture  the  scope  and  impact  of  a 
community-oriented  approach  to  health 
policy. 

It  seeks  to  surface  and  address 
the  overt  and  covert  attitudes  and 
practices,  reinforced  by  both  local 
and  larger  institutions  (such  as 
business,  the  media,  governmental 
entities,  and  special  interests) , 
that  constrain  the  regard,  power  and 
opportunities  accorded  different  age, 
gender,  and  racial/ethnic  social 
status  groups. 

It  simultaneously  acknowledges, 
draws  upon,  invests  in,  and  generates 
the  essential  and  important 
nonmaterial  social  capital  resource 
by  inviting,  listening  to,  and 
maximizing  the  participation, 
involvement,  and  empowerment  of 
individuals  and  groups  within  the 
community  in  defining  priorities  and 
developing  resources  to  address  them. 

It  seeks  to  enlarge  the  human 
capital  assets  and  investments  (such 
as  jobs,  schools,  housing,  and  other 
resources) that  undergird  and  enrich 
the  immediate  and  long-term 
productive  potential  of  those  who 
live,  work,  and  raise  and  care  for 
their  families  in  those 
neighborhoods . 

It  envisions  a  blueprint  and  a 
team  of  architects  —  from  the  public 
and  private  sectors  and  affected 
communities  —  to  undertake  the 
design  of  a  comprehensive, 
integrated,  prevention-oriented 
continuum  or  system  of  programs  and 
services  accessible  to  all  members  of 
the  community. 

It  embraces  a  comprehensive 
definition  of  health  and  well-being, 
and  embodies  a  restive,  normative 
judgment  of  outcomes,  motivated  by 
assessments  of  the  extent  to  which 


the  health  of  the  community  as  a 
whole,  not  just  individual  patients 
or  clients  within  it,  can  be 
improved. 

SUMMARY 

In  summary,  the  perspective 
developed  here  argues  that  ultimately 
the  remedies  for  our  individual  and 
collective  vulnerability  are  found  in 
the  bonds  of  carina;  human 
communities.   The  invitation  to 
understand  and  address  the  health  and 
health  care  needs  of  vulnerable 
populations  may,  in  fact,  take  on  a 
renewed  significance  with  the 
recognition  that  "they"  may,  at  any 
time,  become  "we". 
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Good  morning.   I'm  going  to  report 
to  you  this  morning  on  a  survey  of 
infant  sleeping  position  we  did  in  North 
Carolina.   I'm  going  to  present  this  as 
a  state  health  department  response  to  a 
sudden,  unexpected  data  need.   I  will 
focus  on  two  aspects  of  the  survey:   the 
study  design,  which  may  have  introduced 
bias  into  our  results,  and  how  we 
attempted  to  adjust  for  that;  and  the 
administrative  and  political  context, 
which  affected  the  timing  of  the  survey 
and  its  adequacy  as  a  response  to  an 
unexpected  data  need. 

Our  interest  in  sleeping  position 
begins  with  our  interest  in  sudden 
infant  death  syndrome  (SIDS) .   We  are 
interested  in  SIDS  because  it  is  the 
major  cause  of  postneonatal  mortality, 
and  it  receives  considerable  attention 
in  the  public  and  political  arenas.   As 
a  state  health  department,  there  was 
little  we  could  do  to  prevent  SIDS.   In 
trying  to  understand  SIDS  in  our 
population,  we  note  that  the  rates  have 
been  declining  for  years,  but  the  reason 
for  this  decline  is  unknown. 

Recently,  evidence  began  to 
accumulate  that  placing  the  infant  to 
sleep  on  its  back  or  side  reduces  the 
risk  of  SIDS.   The  evidence  was  strong, 
but  incomplete.   In  North  Carolina,  and 
I  expect  in  most  other  states,  this  was 
met  with  uncertainty  on  the  part  of  the 
public,  and  clinicians  were  reluctant  to 
change  the  recommendations  they  were 
used  to  giving  their  patients.   In  the 
face  of  this,  state  health  officials 
were  hesitant  to  develop  a  policy  on 
sleeping  position  and  to  implement  a 
program  to  effect  that  policy. 

Then  in  June  of  1994,  the  Public 
Health  Service  and  several  other 
agencies  began  a  national  campaign  (the 
"Back-to-Sleep"  campaign) ,  aimed  at 
parents  and  providers,  to  get  parents  to 
place  their  infants  to  sleep  on  their 
backs  or  sides.   State  health  officials 
suddenly  found  themselves  confronted 
with  questions  they  did  not  have  the 
data  to  answer: 

1.    What  is  the  compliance  with  these 
recommendations  in  our  state? 


2. 


What  is  the  effect  of  this  on  SIDS? 


3 .    How  does  this  differ  among  our 
various  population  groups? 

In  order  to  answer  these  questions, 
we  would  have  to  know  current  and  past 
prevalence  of  infant  sleeping  position, 
by  population  group,  so  that  we  could 


assess  past  and  future  changes  in 
sleeping  position  and  relate  these 
changes  in  SIDS.   However,  the  reality 
of  the  situation  was  that  we  had  no 
baseline  data  and  no  control  over  timing 

—  the  national  campaign  was  going  ahead 
whether  we  were  ready  or  not. 
Furthermore,  we  had  no  resources  that 
could  be  immediately  applied  to  this 
problem,  and  we  had  little  local 
experience  to  go  on  as  far  as  this 
particular  issue  goes.   In  addition, 
although  our  researchers  perceived  the 
need  to  collect  this  data  as  soon  as 
possible,  preferably  before  the  start  of 
the  national  campaign,  our 
administrators  did  not  share  that 
perception. 

We  did  eventually  mount  a  survey. 
It's  objective  was  to  collect  baseline 
sleeping  position  data,  including 
"retrospective"  baseline  data,  on  the 
North  Carolina  infant  population  aged  2 
to  4  months.   We  reached  this  population 
by  sampling  birth  certificates  and 
conducting  telephone  interviews  with  the 
mothers.   Questionnaire  items  came  from 
a  survey  that  had  been  conducted  by  the 
National  Institute  for  Child  Health  and 
Human  Development  (NICHD) .   We  asked 
about  sleeping  position  for  the  current 
and  previous  infant.   Our  target 
population  was  the  June  1994  birth 
cohort,  restricted  to  blacks  and  whites 
and  resident  in-state  births.   We  traced 
the  phone  numbers  by  linking  the  birth 
certificates  to  the  health  department 
file  that  contained  information  on  local 
health  department  clients  and  WIC  and 
MEDICAID  clients.   Only  the  infant's 
mother  was  accepted  as  a  respondent. 

Perhaps  the  most  creative  aspect  of 
our  response  to  this  unexpected  data 
need  was  the  way  we  funded  the  survey. 
The  Survey  Operations  Unit  did  not  have 
a  budget  that  could  be  used  for  this 
purpose.   So  we  called  around  to  various 
health  department  programs  —WIC, 
maternal  and  child  health,  injury 
control,  immunizations,  and  adult  health 

—  and  invited  them  to  participate  in  the 
survey  by  providing  questions  for  the 
instrument  and  funds  for  the  budget.   We 
also  received  funds  from  the  Healthy 
People  2000  grant  that  our  center  has 
from  CDC.   In  this  way,  we  assembled  a 
minimum  budget  for  the  survey.   With 
these  funds,  temporary  help  was  hired, 
but  other  resources  were  scarce.   Only 
one  researcher  (J.K.L.)  was  involved  in 
the  project,  and  little  additional 
office  space  or  computers  were 
available.   Our  unit  did  not  have 
interviewers  to  make  the  telephone 
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calls.   One  thing  we  did  have  going  for 
us,  however,  was  wonderful  cooperation 
from  several  agencies  within  the 
department.   The  BRFSS  unit,  for 
example,  did  the  interviews  for  us,  and 
the  print  shop  turned  our  questionnaire 
around  overnight,  even  though  there  was 
a  two  or  three  week  waiting  list.   It 
was  like  that  everywhere  we  turned. 
Without  that  cooperation,  we  could  not 
have  done  the  survey. 

In  evaluating  the  survey  as  a 
response  to  an  unexpected  data  need, 
timing  is  one  of  the  most  important 
issues.   Very  early  in  1994,  we  tried  to 
generate  interest  in  a  sleeping  position 
survey,  but  that  was  moving  along 
slowly.   Then  around  March,  we  learned 
of  the  imminence  of  the  national 
campaign.   At  that  point,  we  began 
recruiting  resources  in  earnest.   By 
June,  the  first  memos  had  been  written. 
In  August,  final  authorization  was 
received,  funding  was  finalized, 
temporary  help  was  hired,  the 
questionnaire  was  finalized,  and  the 
sampling  frame  was  received.   The  sample 
was  selected  in  September,  and  we 
verified  (by  hand!)  that  these  were  not 
cases  of  infant  death  or  adoption. 
Following  a  pilot  study,  telephone 
numbers  were  traced  and  interviews 
conducted  in  September  -  November. 

Figure  1  shows  the  development  of 
our  sample.   There  were  7,839  eligible 
births  in  the  June  cohort,  but  only 
5,496  of  these  were  included  in  the 
sampling  frame  we  received  from  Vital 
Records.   The  remaining  2,343  births  had 
not  yet  been  reported  to  Vital  Records. 
We  could  have  begun  with  a  more  complete 
frame  by  using  the  May  cohort,  but  that 
would  have  given  us  an  infant  population 
older  than  what  we  were  aiming  at,  so  we 
decided  to  go  with  the  incomplete  frame. 
Of  the  sample  that  was  subsequently 
selected,  about  half  were  not  contacted, 
primarily  because  correct  telephone 
numbers  were  not  obtained  for  them. 
Nearly  everyone  contacted  agreed  to  an 
interview. 

Figure  1  indicates  two  potential 
sources  of  bias  —  the  incomplete 
sampling  frame  and  the  high  rate  of 
noncontact.   We  adjusted  for  these  using 
standard  methods  of  poststratif ication 
based  on  data  from  the  birth 
certificates,  which  we  had  on  the  entire 
June  cohort,  not  just  those  interviewed. 
The  incomplete  sampling  frame  was 
poststratif ied  by  hospital  of  birth  and 
race.   The  contacted  sample  was 
poststratif ied  by  race,  maternal 
education,  and  maternal  age.   I  can 


provide  further  details  of  these 
adjustments  if  you  wish. 

Table  1  shows  the  prevalence  of  the 
recommended  sleeping  position  by  race 
and  sex  for  the  current  and  previous 
birth.   Previous  birth  here  represents 
about  2  to  4  years  ago.   At  that  time, 
there  was  little  difference  in 
prevalence  between  blacks  and  whites  — 
all  were  at  around  35  percent.   For  the 
current  birth,  both  groups  show 
increased  use  of  the  recommended 
position,  but  whites  have  increased 
slightly  more  than  blacks. 


Table  1.   Prevalence  (%)  of  back  or  side 
sleeping  position 


Race 

Males 
Current  Birth 

Females 

Blacks 

45.8 

48.1 

Whites 

59.0 
Previous  Birth 

53.4 

Blacks 

36.1 

33.8 

Whites 

38.8 

36.7 

What  can  be  learned  from  our 
experience?   Regarding  the  study  design: 
the  most  important  thing  was  to  be  able 
to  act  quickly.   Using  birth 
certificates  as  the  sampling  frame 
enabled  us  to  do  this.   The  birth 
certificates  were  immediately  available 
to  us  at  no  cost,  and  they  gave  us 
access  to  the  general  population.   They 
also  provided  a  considerable  amount  of 
data  on  the  entire  target  population  at 
zero  marginal  cost.   All  of  these  were 
distinct  advantages  to  us.   On  the  other 
hand,  because  of  the  time  needed  between 
obtaining  the  sampling  frame  and 
conducting  the  interviews,  we  had  to  use 
an  incomplete  frame,  which  may  have 
introduced  bias.   In  addition,  since  we 
relied  on  tracing  telephone  numbers 
based  on  birth  certificate  information, 
we  had  a  high  rate  of  noncontact  due  to 
untraced  numbers.   This  also  may  have 
introduced  bias. 

In  the  future,  a  complete  sampling 
frame  could  be  achieved  by  sampling  and 
interviewing  in  two  stages,  i.e.,  as  we 
did,  and  again  some  weeks  later  when  the 
rest  of  the  births  have  been  reported. 
For  our  purposes,  this  would  have  given 
an  infant  population  that  was  older  than 
we  needed.   Alternatively,  it  may  be 
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possible  to  conduct  a  brief  supplemental 
survey  of  the  omitted  part  of  the  frame 
to  collect  information  to  use  in 
adjusting  for  bias.   However,  my 
preference  would  be  to  concentrate 
resources  at  an  early  stage  of  the 
survey  to  shorten  the  lead  time  and 
thereby  be  able  to  start  with  a  more 
complete  frame. 

Regarding  the  high  rate  of 
noncontact,  it  was  not  feasible  for  us 
to  conduct  a  mail  survey  of  those  for 
whom  we  did  not  have  telephone  numbers. 
However,  there  were  several  instances 
where  the  number  we  had  was  to  a 
relative  (the  respondent  not  having  a 
telephone  at  her  residence) ,  we  left  a 
message,  the  respondent  called  us  back, 
and  we  obtained  an  interview  that  way. 
This  leads  me  to  believe  that,  in  the 
future,  we  should  mail  letters  to  those 
for  whom  we  do  not  have  telephone 
numbers,  inviting  them  to  call  in  for  an 
interview.   In  this  way,  we  may  be  able 
to  increase  coverage. 

When  we  first  began  to  plan  the 
survey,  we  were  not  at  all  certain  that 
we  would  be  able  to  pull  it  off  in  time 
and  obtain  useful,  important  data.   It 
would  have  been  better  if  we  could  have 
collected  the  data  before  the  start  of 
the  national  campaign.   Nonetheless,  our 
results  give  important  information  that 
will  be  used  to  guide  the  current  SIDS 
prevention  program  in  North  Carolina  and 
to  evaluate  future  changes  in  SIDS. 

There  is  a  lesson  here  for 
administrators.   That  is  that  it  may  be 
necessary,  in  the  interest  of  the 
public's  health,  to  embark  on  a 
nontraditional  data  collection  effort 
before  it  is  politically  comfortable  to 
do  so.   Since  we  can  expect  such 
instances  to  become  increasingly  common, 
mechanisms  should  be  established  to 
anticipate  unexpected  data  needs  and 
respond  to  them  as  they  arise. 


June  1W4 

Birth  Cohort 

n=  7,839 


Included  in 

Sampling  Frame 

n=  5,496 


Selected  for 

Sample 

n=  2,494 


Contacted  by 
Telephone 
n=  1,191 


Interviewed 
n=  1,143 


Omitted  from 

Sampling  Frame 

n=  2,343 


Not  Selected 
n=  3,002 


Not  Contacted 

n=  1,303 
ineligible  - 19 

moved -24 

no  answer  -  47 

phone  number 

not  obtained-  1,213 


Not  Interviewed 

n  =  48 

refused-  40 

language  barrier  -  5 

other  -  3 


Figure  1.  The  1994  North  Carolina  Current  Birth  Cohort  Survey. 
Development  of  the  Study  Population. 


Thank  you. 
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BIRTH  PRODUCT  LINE  QUALITY  MANAGEMENT  REVIEW 


David  C.  Schutt,  Colonel,  USAF,  MC,  Department  of  Defense 

Raymond  S.  Crawford,  III,  MD,  MBA 

Henry  Krahauer,  CAPT,  MD,  Ph.D.,  PHS 


BACKROUND 

The  Military  Health  Services  System  (MHSS) 
has  8.25  million  beneficiaries  world  wide. 
These  beneficiaries  are  cared  for  by  50,000 
providers  in  120  hospitals  and  over  500  clinics. 
The  MHSS  generates  a  million  admissions  a  year 
and  50  million  outpatient  visits  a  year. 

The  MHSS  is  under  the  same  pressure  to 
change  as  civilian  medicine.   That  is  to 
decrease  costs  while  maintaining  quality. 
Managed  care  seems  to  provide  some  of  the  tools 
to  accomplish  this  transition.   In  addition,  the 
Department  of  Defense  is  using  capitation  to 
budget  for  health  care.   The  MHSS  is 
implementing  both  of  these  in  a  managed  care 
program  called  TRICARE. 

Births  are  the  most  common  reason  to  be 
admitted  to  a  military  hospital.   Therefore, 
births  were  chosen  as  the  first  "product  line". 
The  Birth  Product  Line  analyzes  the  clinical 
outcomes,  processes  of  care  and  resource 
utilization  associated  with  the  care  of  9700 
mothers  and  newborns  from  the  first  prenatal 
visit  to  discharge.   This  analysis  results  in 
the  production  of  a  Clinical  Practice  Profile 
which  elucidates  best  clinical  practice. 

GOALS 

The  goals  of  the  Birth  Product  Line  study 
were: 

•  Determine  the  influence  of  patient 
characteristics  and  processes  of  care  on 
clinical  outcomes 

•  Determine  the  influence  of  patient 
characteristics  and  processes  of  care  on 
resource  utilization 

•  Develop  Clinical  Practice  Profiles  by 
hospital  that  describe  performance  in  terms 
of  clinical  outcomes  and  resource  use 

•  Identify  the  processes  of  care  that  lead  to 
desired  clinical  outcomes  -  Best  Clinical 
Practice 

•  Disseminate  this  information  to  the  MHSS 

•  Enable  providers  to  incorporate  Best  Clinical 
Practice  patterns  in  their  hospitals 

BEST  CLINICAL  PRACTICE 

Best  Clinical  Practice  is  the  combination 
of  processes  of  care  that  produce  a  optimal 
clinical  outcome  with  the  most  cost  effective 
use  of  resources.   This  concept  can  be  visualize 
as  follows: 

BEST  CLINICAL  PRACTICE=OPTIMAL  CLINICAL  OUTCOME 

COST  EFFECTIVE  RESOURCES 


STUDY  RATIONALE 

Births  are  the  most  common  reason  to  be 
admitted  to  a  military  hospital  and  are  the 
largest  generator  of  beddays .  Pregnancy  and 
child  birth  are  the  most  common  reason  for  a 
malpractice  claim  and  as  more  women  join  the 
active  duty  force,  women's  health  care  issues 
become  a  greater  concern. 

The  Birth  Product  Line  study  will  enable 
military  hospitals  to  improve  the  quality  of 
care  for  mothers  and  newborns  while  reducing 
clinical  risk  and  costs. 

The  DoD  has  now  gone  to  a  capitated  funding 
model  for  health  care.   This  model  switches  the 
funding  philosophy  from  a  "fee  per  service"  to  a 
"fee  per  person"  approach.   In  order  to  live 
within  these  new  constrained  budgets,  providers 
are  now  asking  four  questions: 

•  Which  processes  of  care  should  we  stop  doing? 

•  Which  processes  of  care  should  we  continue 
doing? 

•  Which  processes  of  care  should  we  keep  doing 
but  improve? 

•  Which  new  processes  of  care  should  we  start 
doing? 

The  birth  product  line  study  attempts  to 
begin  a  methodology  to  answer  the  clinicians 
questions . 

STVPX  PES TGN, 

One  hundred  births  were  randomly  sampled  at 
97  military  hospitals.   The  cases  were  followed 
from  the  first  prenatal  visit  to  discharge  of 
mother  and  baby.   The  cases  reviewed  were 
delivered  between  October  1992-September  1993. 

The  analysis  of  the  data  was  done  in  five 
stages : 

•  Establish  patient  risk  adjustment  factors 

•  Determine  the  processes  of  care  delivered  to 
the  patient 

•  Determine  the  clinical  outcomes  and  assign  a 
severity  score 

•  Establish  a  resource  use  model 

•  Determine  influence  of  processes  of  care  on 
clinical  outcomes  and  resource  use 

One  hundred  nine  risk  factors  were 
identified  by  clinicians  which  might  affect  the 
outcome.   They  also  identified  47  processes  of 
care  which  were  delivered  to  the  patients. 

Each  clinical  outcome  was  assigned  a 
severity  score  of  0-12  where  0  is  no  abnormality 
or  finding  and  12  is  death  (Figure  1).   Each 
clinical  outcome  is  a  compost  of  separate 
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Figure  1 
How  Combined  Outcomes  are  Developed  from  Individual  Findings 
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findings  such  as  extent  of  blood  loss  or 
respiratory  complications.   These  findings  were 
assembled  into  pathophysiologic  clusters.   The 
clusters  were  grouped  to  form  intermediate 
maternal,  e.g.,  genital  tract  trauma, 
infections,  and  neonatal  outcomes,  e.g., 
prematurity,  infections,  trauma,  etc.   The 
intermediate  outcomes  were  combined  to  form 
aggregate  maternal  and  aggregate  neonatal 
outcomes  and  finally  these  two  were  combined  to 
form  the  Combined  Maternal  and  Neonatal  Outcome. 

Resource  Cost  Units  were  developed  for 
maternal  utilization,  neonatal  utilization  and  a 
combined  utilization  (Figure  2).   The  cost  units 
have  two  components:  (1)  physician  charges  for 
services  and  procedures  and  (2)  hospital  charges 
for  services  provided.   Physician  charges  were 
calculated  using  ICD-9-CM  codes  and  CPT-4  codes. 
Hospital  charges  were  calculated  using  CHAMPUS 
DRGs. 


Figure  2 
How  Resource  Cost  Units  Are  Developed 
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FINDINGS 

Figure  3,  a  Clinical  Practice  Profile 
(CPP),  attempts  to  solve  the  best  clinical 
practice  relationship,  e.g.,  optimal  clinical 
outcome  with  the  most  effective  use  of 
resources.   The  x  axis  displays  the  difference 
in  the  predicted  resource  use  and  actual 
resource  use.   The  y  axis  displays  the 
difference  in  the  predicted  outcome  and  the 
actual  outcome.   Best  clinical  practice  is  based 
on  the  practice  patterns  of  hospitals  that 
currently  produce  optimal  clinical  outcomes  at 
the  most  effective  use  of  resources.   Figure  3 
shows  using  a  CPP  the  combined  maternal  and 
neonatal  clinical  outcome  performance  and 
resource  utilization  for  outlier  hospitals. 
Hospitals  are  outliers  if  their  clinical 
outcomes  and  resource  utilization  are  better  or 
worse  than  predicted.   The  five  hospitals 
identified  with  dots  are  the  five  with  the  best 
clinical  practice. 

As  an  example,  look  at  Hospital  E.   In 
Figure  3  Hospital  E  is  in  the  lower  right  hand 
quadrant.   Their  combined  maternal  and  neonatal 
outcomes  are  not  a  good  as  predicted  and  their 
resource  utilization  is  higher  than  predicted. 
How  does  the  rate  at  which  they  use  certain 
processes  of  care  compare  with  the  rest  of  DoD? 
Table  1  provides  the  answer.   For  example, 
diabetes  screening  lowers  or  improves  the 
outcome  score  by  .27  units.   It  lowers  the 
resource  utilization  by  .04  units.   Hospital  E 
performs  diabetes  screening  79%  of  the  time 
while  the  DoD  on  average  performs  the  screen  82% 
of  the  time. 

The  five  best  hospitals  are  in  the  left 
upper  quadrant  of  Figure  3 .   How  often  do  they 
perform  this  screen  and  how  does  Hospital  E 
compare?   Table  1  shows  in  the  far  right  hand 


Figure  3 
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Tabic  1 

Births 
Impact  of  Processes  of  Care  on  Combined  Outcomes  and  Resource  Utilization 


Process  of  Care' 


Change  In  Proportional  Hospital  E 

Outcome  Change  In 

Severity  Resource  How  Often    How  Often 

Score  P  Value         Cost  Unit  P  Value  Performed     Performed 


MTPt  wtlli 

Better  Than 

Expected 

DoO    Performance 


How  Often 
Performed 


Pre-admission  Interventions 

CxtemaJ  version  attempted,  antepartum 

Diabetes  screen  performed 

Total  number  of  prenatal  visits  (average,  per  patient) 

Number  of  ultrasound  tests  performed  (average,  per  patient) 

Ultrasound  testing  performed 

PTe-dellrery  Interventions 

External  fetal  monitoring  only 

Artificial  rupture  of  membranes 

Internal  uterine  pressure  catheter  monitoring 

Epidural  anesthesia  for  labor 

Fetal  scalp  pH  tested  during  labor 

Delivery  Interventions 

Outlet  or  low  forceps  extraction 
Non-elective  cesarean  section 


0  57 

0.06 

0.12 

0.00 

2.02% 

-0.27 

0.01 

-0.04 

0.00 

76.79% 

-0.04 

0  00 

0  00 

0.01 

10  0 

Oil 

0.00 

0.04 

0  00 

26 

0.42 

002 

0.04 

0.01 

93.94% 

-0.30 

0.00 

004 

0  00 

46.46% 

-026 

0.00 

-0.05 

0.00 

44  44% 

0.40 

0.00 

0  04 

0  00 

33.33% 

0.61 

0.00 

0.10 

ooo 

59.60% 

160 

0.00 

0  10 

0.06 

10.10% 

1  20 

000 

0  10 

000 

22.22% 

1.98 

0.00 

0.41 

0.00 

16  16% 

I  50% 

82  30% 

119 


19  60* 
53  20% 
32.90% 
19.10% 
0  60% 


4  80% 
10.60% 


1  60% 

87.50% 

11.0 


:n  «0 

50.40% 
26.00% 
11  10% 
000% 


1.70% 
11.60% 


Processes  of  care  with  p  values  <0  05  are  considered  statistically  significant 
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column  that  in  the  five  best  hospitals  diabetes 
screening  is  performed  almost  88%  of  the  time. 

If  Hospital  E  performed  all  of  the 
processes  of  care  evaluated  in  the  study  at  the 
same  rate  as  the  top  five  hospitals,  how  much 
improvement  would  Hospital  E  experience?  The 
thlTL13   in.Fi9ure  4-   The  financial  impact  of 
this  change  in  practice  pattern  is  displayed  in 
Table  2.   if  Hospital  E  performed  these 
processes  of  care  at  the  same  rate  as  the  top 
five  hospitals,  Hospital  E  could  save  $1.4M 
while  maintaining  or  improving  the  technical 
quality  of  care.   The  entire  DoD  could 
experience  a  savings  of  $38M  if  all  of  its 
hospitals  performed  the  way  the  top  five 
performed . 


Figure  4 
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Table  2 


Change  in  Outcome  Severity  Scores  and  Relative  Cost  Units 
Based  on  the  Practice  Pattern  of  the  Five  Best  MTFs 


Change  In 

Change  In 

Outcome 

Relative 

Total 

Severity 

Cost 

Total 

Savings  Per 

Savings 

Organization  Unit 

Score 

Unit 

Cases 

Case  (S) 

(Millions  51 

Hospital  E 

-1.00 

-52.75 

858 

-1,661.63 

1.43 

DoD 

-0.31 

-16.90 

71,037 

532.42 

37.82 
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NEXT    STEP 

Over  the  next  two  years  the  program  will: 

•  Begin  to  answer  the  clinicians'  four 
questions 

Disseminate  the  findings  of  this  study  to  the 
clinicians 

•  Repeat  the  study  in  FY96  to  include  the 
civilian  hospitals  in  the  DoD  network 

•  Offer  support  to  hospitals  in  the 
interpretation  of  the  studies  and  the 
implementation  of  its  findings 
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METHODOLOGY  FOR  THE  EVALUATION  OF  THE  NEW  JERSEY  HEALTHSTART  PROGRAM- 
HEALTH  OUTCOMES  OF  COMPREHENSIVE  MATERNITY  AND  PEDIATRIC  PREVENTIVE  HEALTH  CARE 

Maryanne  Florio,  New  Jersey  State  Department  of  Health 
Susan  Lenox  Goldman,  and  Leah  Ziskin 


Introduction 

In  1988  the  New  Jersey  State 
Departments  of  Health  ( NJDOH )  and  Human 
Services  ( N JDHS )  initiated  HealthStart, 
a  Statewide  enhancement  of  Medicaid 
comprehensive  maternity  and  pediatric 
preventive  health  care  services.   The 
goals  of  this  program  were  to  improve 
access  to  quality  health  care  for  low 
income  pregnant  women  and  children 
to  the  age  of  two,  with  a  resulting 
improvement  in  health  outcomes, 
especially  infant  mortality,  low  birth 
weight,  and  immunization  rates. 
Key  Features  of  the  HealthStart  Program 
The  key  features  of  the  NJ  Health- 
Start  program  include  provision  of  an 
enriched  comprehensive  package  of 
medical  maternity  and  health  support 
services  with  case  coordination; 
pediatric  preventive  health  care 
services;  increased  provider  reim- 
bursement; and  Medicaid  expansion 
through  Jersey  Care  and  presumptive 
eligibility. 

The  health  support  component  is  the 
unique  aspect  of  HealthStart  maternity 
services.   It  consists  of  assessment, 
guidance,  counseling,  and  referral   for 
nutrition,  social,  psychological  needs 
and  support,  and  health  education,  with 
an  individualized  plan  of  care  and  case 
coordination  of  all  services.   Special- 
ized services,  basic  and  extensive 
laboratory  testing,  and  home  visits  for 
high  risk  patients  may  be  provided 
directly  or  by  referral  to  agencies  with 
whom  collaborative  agreements  have  been 
made.   Case  coordination  is  considered 
the  key  component  of  the  health  support 
services  component. 

The  comprehensive  packages  of 
maternity  services  are  available  at 
HealthStart-certified  hospitals, 
community-based  agencies,  and  private- 
practice  settings.   Published  standards 
of  care  and  guidelines,  developed  and 
revised  with  involvement  of  community- 
based  agencies,  help  to  ensure  the 
consistent  delivery  of  quality  care  at 
all  provider  sites.   Medicaid  providers 
who  apply  and  meet  the  criteria,  become 
certified  by  the  Department  of  Health  as 
HealthStart  providers.   Currently,  there 
are  260  certified  medical  practices 
providing  HealthStart  medical  maternity 
care  and  108  certified  comprehensive 
maternity  care  sites  providing  the 
health  support  services. 

HealthStart  pediatric  services 
include  nine  preventive  child  health 
visits  in  two  years  and  age-appropriate 
immunizations  in  accordance  with  the 
national  recommendations.   Providers  are 
also  required  to  provide  or  arrange  for 
outreach,  counseling,  sick  care,  and 
referrals,  where  appropriate. 


HealthStart  services  are  provided 
to  an  increasingly  larger  population  of 
poor  women  by  expanding  eligibility,  to 
100%  of  the  Federal  poverty  level  in 
1988,  and  to  185%  in  1991;   and  through 
presumptive  eligibility  which  allows  a 
woman  to  be  determined  eligible  by  the 
provider  and  receive  services  while 
waiting  for  her  Medicaid  eligibility  to 
become  approved . 
Evaluation  Methodology 

The  evaluation  of  the  HealthStart 
program  was  mandated  as  part  of  the 
HealthStart  legislation,  and  planned  and 
developed  prior  to  the  implementation  of 
the  program.   The  ability  to  conduct  an 
evaluation  was  enhanced  by  this  early 
planning  and  legislation.   The  Health- 
Start  evaluation  was  initiated  to  assess 
the  effectiveness  of  the  program  in 
terms  of  both  process  and  the  outcome, 
and  addressed  five  specific  research 
question:   how  successful  has  the 
HealthStart  program  been  in  1 )  improving 
access  to  quality  maternity  and  ped- 
iatric health  care  services  for  low 
income  pregnant  women  and  children, 
2)  providing  delivery  of  quality 
comprehensive  services  by  certified 
providers,   3)  reducing  barriers  to  the 
delivery  of  services  to  this  population, 
including  language  and  cultural  bar- 
riers, 4)   improving  health  outcomes  of 
participating  women  and  children,  and 
5)   providing  a  cost-effective  package 
of  services? 

A  six-year  evaluation  study  has 
been  conducted  of  this  program.   NJ  has 
used  several  methodologies  to  determine 
the  impact  of  the  HealthStart  program. 
Data  Collection  Methodology 
HealthStart  Linked  Data  System: 

The  key  methodology  used  in  the 
evaluation  of  HealthStart  is  a  linkage 
of  five  databases:   1,2)  NJDOH  Vital 
Statistics  birth  certificate  files 
matched  to  the  infant  death  certificate 
files,  3)  NJDOH  hospital  uniform  billing 
discharge  files  (UB  82),  and  4,5)  NJDHS 
Medicaid  claims  and  eligibility  files. 
This  linked  data  system  (HSLDS)  consists 
of  individual  records  for  all  single, 
live,  in-hospital  births  to  NJ  resi- 
dents.  The  number  of  single  live  NJ 
births  on  the  HSLDS  was  89,500  in  1985, 
and  increased  each  year,  to  113,300  in 
1990.   Medicaid  births  represented 
approximately  15%  of  these  births  in 
1985,  and  increased  to  20%  by  1990. 
HealthStart  accounted  for  3%  of  the 
Medicaid  population  in  its  first  year, 
1988,  and  increased  to  65%  by  1990. 
At  the  present  time,  the  HealthStart 
program  serves  approximately  90%  of  the 
Medicaid  population  of  pregnant  women 
and  young  children. 

The  HSLDS  also  contains  a  medical 


165 


history  profile  of  all  mothers  from 
three  months  prior  to  conception  through 
three  months  after  delivery,  and  a  com- 
parable medical  profile  for  all  children 
up  to  the  age  of  36  months.   Also,  the 
files  are  linked  across  all  years  since 
1985  so  that  a  woman's  entire  pregnancy 
history  for  all  of  her  pregnancies  and 
medical  history  is  linked. 

The  linkage  of  the  files  for  the 
baseline  years  of  1985  through  1987,  and 
the  first  three  years  of  the  HealthStart 
program,  1988  through  1990,  has  been 
completed  through  a  collaborative  effort 
with  the  NJ  Department  of  Treasury, 
Office  of  Telecommunication  and  Infor- 
mation Services  at  a  93%  to  96%  match 
rate  across  all  files,  and  includes  100% 
of  the  birth  records  in  NJ  each  year. 
The  linkage  is  nearly  completed  for  1991 
and  1992,  and  is  anticipated  to  be  done 
for  1993  in  the  near  future.   The  NJ  WIC 
data  system  is  also  being  linked  to  the 
HealthStart  Linked  Data  System  (HSLDS) 
at  the  present  time. 

This  unique  data  linkage  enables  NJ 
to  make  extensive  comparisons  of  health 
outcomes  and  costs  for  women  and  chil- 
dren who  receive  HealthStart  services 
with  other  Medicaid  pregnant  women  and 
their  children,  and  with  all  other 
pregnant  women  and  young  children  in  NJ 
who  did  not  receive  Medicaid  funds. 
Although  the  HSLDS  system  has  been 
costly,  time  and  staff  intensive,  and 
has  experienced  delays  because  of  major 
changes  in  the  birth  certificate  file, 
and  the  Medicaid  claims  and  eligibility 
system,   it  has  been  an  invaluable  tool 
in  the  evaluation  of  HealthStart  and  in 
providing  information  on  maternal  and 
child  health  issues  in  NJ. 
HealthStart  Maternity  Services  Summary 
Data: 


A  second  major  source  of  data  is 
an  in-depth  collection  of  pregnancy- 
related  information  maintained  via  the 
HealthStart  Maternity  Services  Summary 
Data  (MSSD)  form  since  the  initiation  of 
the  program  for  each  woman  receiving 
HealthStart  services.   It  is  currently 
available  through  1993  for  each  woman 
receiving  HealthStart  services,  and 
includes  demographics,  risk  factors, 
services  received,  pregnancy  outcome, 
and  health  status  of  the  mother  and  her 
newborn.   These  data  were  available 
early  in  the  evaluation  and  were  used 
initially  for  preliminary  process  and 
evaluation  data,  and  provider  quality 
assurance.   At  present  they  are  used  to 
produce  indicators  of  effective  program 
implementation,  access  to  and  provision 
of  services,  and  confirmation  and 
extension  of  the  linked  data  system  to 
more  recent  years . 
Pediatric  Preventive  Health  Care  Data: 

Third,  providers  of  HealthStart 
pediatric  services  submit  an  individual 
patient  report  form  for  each  pediatric 
preventive  health  care  visit.   The  data 
used  in  the  analysis  of  the  HealthStart 
pediatric  program  are  based  on  375,000 
visits  by  175,00  children  ( unduplicated ) 
under  the  age  of  two  years. 


HealthStart  Provider  Tracking  System: 

The  fourth  major  data  source  is  the 
Provider  Monitoring  System,  a  system  of 
information  on  each  provider,  including 
certification,  technical  assistance 
visits,  staffing  patterns,  and  recerti- 
fication.   It  has  been  developed  to 
assist  in  the   monitoring,  quality 
assurance,  technical  assistance,  and 
certification  of  the  HealthStart 
provider  agencies. 
Limitations  of  Data 

As  with  all  data  there  are 
limitations  to  these  data  sets: 
1)  portions  of  the  data  are  self- 
reported  and  therefore,  some  of  the 
sensitive  data,  for  example  drug  use, 
may  be  underreported;   2)  there  is  a 
chance  of  error  in  completing  forms, 
in  data  entry,  or  in  data  processing 
(extensive  systems  of  edit  checks  have 
been  developed  to  control  for  data 
errors ) ;   3 )  some  of  the  data  are  incom- 
plete;  (The  number  of  MSSD  forms 
submitted  is  not  equal  to  the  number 
served.   This  appears  to  be  random  in 
all  but  one  agency  who  submit  their 
forms  electronically,  and  experienced  a 
delay  because  the  electronic  system  was 
not  implemented  until  1991.   Also,  not 
all  of  the  records  were  linked  in  the 
LDS.   The  percentage  of  non-linked 
records  is  higher  in  the  Medicaid  pop- 
ulation, but  this  appears  to  be  a  random 
non-linkage  of  HealthStart  and  non- 
HealthStart  Medicaid  women.)   4)  there 
is  a  potential  selection  bias  for 
HealthStart  women,  i.e.,  the  more 
motivated  or  healthier  women  may  have 
chosen  to  participate  in  the  program, 
which  could  potentially  result  in 
healthier  outcomes  due  to  differences 
other  than  those  due  to  the  impact  of 
the  HealthStart  program.   (There  are 
three  ways  in  which  this  bias  has  been 
tested.   First,  HealthStart  women  were 
found  to  be  nearly  identical  to  other 
Medicaid  women  in  the  observ- 
able characteristics:  age  (33%  teens,  3% 
over  34),  race/ethnicity  (60%  black,  30% 
Hispanic),  marital  status  (80%  not 
married),  and  education  (50%  had  less 

than  high  school)  .   Secondly,  the 
results  of  a  survey  of  all  women  who 
received  HealthStart  services  showed 
that  availability  was  the  predominant 

2 
determinant  of  participation  .   Lastly, 

a  study  conducted  by  an  independent  pop- 
ulation and  economic  researcher  showed 
that  women  who  participated  in  Health- 
Start  were  actually  more  at  risk  for 
adverse  outcomes  than  other  Medicaid 

4 
women  . 

Confirmation  of  Data 

Checks  of  key  sensitive  and  self- 
reported  data  have  been  made  with  data 
from  other  studies.   For  example,  checks 
were  made  of  self-reported  data  on  drug 
use  by  HealthStart  women  by  comparing 
their  data  with  data  from  a  statewide 
drug  study  on  pregnant  women,  which 
showed  that  HealthStart  self-reported 
data  were  approximately  five  percentage 
points  lower  than  the  drug  study  data. 
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Outcome  Data  and  Findings  of  the 
HealthStart  Evaluation 

The  results  of  the  evaluation  show 
that  the  HealthStart  program  has  been 
successful  in  providing  access  to 
enhanced  quality  health  care  services 
and  improving  health  outcomes  for  low 
income  pregnant  women  and  children.   The 
most  significant  results  of  the 
provision  of  the  HealthStart  model  are 
listed  below  as  answers  to  the  five 
principle  research  questions. 
Increased  Access  to  Health  Care: 
o In  1988  the  HealthStart  agencies 
provided  access  to  the  enhanced 
package  of  services  to  8,000  women. 
The  HealthStart  program  is  now 
providing  prenatal  care  services  to 
more  that  90%  of  the  population  of 
pregnant  Medicaid-eligible  women, 
approximately  35,000  women  at  108 
comprehensive  maternity  provider 
sites,  of  whom  approximately  one-half 
deliver  in  a  given  year; 
o   In  1988  10,500  pregnant  women 

received  HealthStart  services  through 
the  presumptive  eligibility  process; 
by  1993,  this  increased  to  23,700 
women; 
o  Entry  into  prenatal  care  in  the  first 
trimester  increased  from  33%  in  1988 
to  40%  by  19922; 
o   In  1988  HealthStart  pediatric 

preventive  health  care  was  provided 
to  6,600  children;  it  is  now  provided 
to  approximately  38,000  children  a 

year  by  more  than  300  providers3; 


Provision  of  Enhanced  Quality  Prenatal 
Care  Services; 

o   HealthStart  agencies  provide  the 
entire  package  of  enhanced  prenatal 
care  services  to  95%  to  100%  of  the 
women  who  go  to  a  HealthStart 

agency  ; 
o  HealthStart  women  and  children 

receiving  WIC  increased  from  65%  in 

1988  to  80%  by  19922; 

Improved  Health  Outcomes  for  Newborn  of 

HealthStart  Mothers: 

o  Fewer  black  (non-Hispanic)  babies  of 
mothers  who  received  HealthStart 
services  died  in  their  first  27  days 
of  life  (neonatal  mortality)  than 
babies  of  other  black  (non-Hispanic) 
Medicaid  women,  and  surprisingly, 
than  black  (non-Hispanic)  babies  of 
non-Medicald  women  (7.5,  7.9,  and  8 . 7 
deaths  per  thousand  live  births  to 
HealthStart  women  compared  to  8.5, 
12.1  and  10.9  deaths  per  thousand 
live  births  to  Medicaid  non- 
HealthStart  women,  and  11.5,  10.4, 
and  10.5  deaths  per  1000  live  births 
to  non-Medicaid  women  in  1988,  1989, 

and  1990  respectively,  p  <  .0001  , 
Figure  1); 
o  Fewer  black  (non-Hispanic)  very  low 
birth  weight  babies  (less  than  1500 
grams)  were  born  to  women  who 
received  HealthStart  services  than  to 
other  black  (non-Hispanic)  Medicaid, 
and  again  surprisingly,  non-Medicaid 
pregnant  women  (1.34%,  1 .45%,  and 
1 . 58%  of  HealthStart  newborn  had  very 


Figure  1:    Neonatal  Mortality  (0  to  27  Days) 

Comparison  of  Black  Non-Hispanic  Newborn 

Deaths  per  1000  Live  Births 
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low  birth  weight  compared  to  2.12%, 
2.67%,  and  2.72%  of  Medicaid  non- 
HealthStart  newborn,  and  2.40%, 
2.64%,  and  2.80%  for  non-Medicaid 
newborn  in  1988,  1989,  and  1990 

respectively,  p  <  .001  ,  Figure  2); 
Newborn  of  women  who  received 
HealthStart  prenatal  care  services 
had  significantly  lower  infant 
mortality,  especially  neonatal 
mortality  than  newborn  infants  of 
other  Medicaid  women  (6.2,  5.6,  and 
6.3  neonatal  deaths  per  1000  live 
births  for  HealthStart  newborn 
compared  to  7.3,  8.3,  and  9 . 1  deaths 
for  Medicaid  non-HealthStart  women  in 
1988,  1989,  and  1990  respectively,  p 

<  .00011); 

The  percentage  of  moderately  low 
birth  weight  newborn  of  women  who 
received  HealthStart  prenatal  care 
services  decreased  from  8.9%  in  1988 
to  6.7%  in  1992;   also,  a 
significantly  lower  percentage  of 
moderately  low  birth  weight  babies 
were  born  to  HealthStart  women  than 
to  other  Medicaid  women  (7.9%,  7.9%, 
and  7.2%  of  HealthStart  women  had 
moderately  low  birth  weight  newborn 
compared  to  8.3%,  10.0%,  and  9.4%  of 
other  Medicaid  women  in  1988,  1989, 

and  1990  respectively,  p  <  .005)  ) ; 
The  percentage  of  very  low  birth 
weight  newborn  of  HealthStart  women 
decreased  from  2.0%  in  1988  to  1.1% 
in  1992;  also,  a  significantly  lower 
percentage  of  very  low  birth  weight 


babies  were  born  to  HealthStart  women 
than  to  other  Medicaid  women  (1.0%, 
1.2%,  and  1.2%  of  HealthStart  women 
had  very  low  birth  weight  newborn 
compared  to  1.6%,  1.9%,  and  1.9%  of 
other  Medicaid  women  in  1988,  1989, 

and  1990  respectively,  p  <  .005  ); 
The  percentage  of  well  newborn  in- 
fants of  HealthStart  women  increased 

from  71%  in  1988  to  84%  by  19922; 
There  was  a  comparable  decrease  in 
the  percentage  of  HealthStart  newborn 
infants  with  medical  problems,  the 
most  common  of  which  were  respiratory 
distress,  sepsis,  apnea,  cardiac 
distress,  and  drug  withdrawal,  from 

15%  in  1988  to  7%  in  19922; 
There  was  a  decrease  in  the  use  of 
the  neonatal  intermediate  or  special 
care  nursery  from  15%  in  1988  to  9% 

in  19922; 

There  was  a  decrease  in  the  percen- 
tage of  HealthStart  infants 
hospitalized  since  birth  or  rehos- 
pitalized  from  .28%  in  1988  to  .06% 

in  19922; 

There  was  a  high  age-appropriate 
immunization  rate  of  79%  for  the 
three  most  important  childhood 
immunizations  for  children  completing 
HealthStart  pediatric  services  (DPT, 
polio,  and  MMR)  when  compared  with 
the  Statewide  average  immunization 

rate  of  40%  to  50%3; 


Figure  2:  Very  Low  Birth  Weight  (Less  than  1500  grams) 
Comparison  of  Black   Non-Hispanic   Newborn 

Percentage  of  Newborn 
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Improved  Health  Outcomes  for  the 
HealthStart  Mother: 

o  Substance  abuse  decreased  during 
pregnancy:   drugs  from  5%  to  1%, 
alcohol  from  6%  to  2%,  and  smoking 
from  24%  to  12%  by  the  time  of 

delivery  ; 
o  The  percentage  of  uncomplicated  labor 
and  deliveries  increased  from  59%  in 

1988  to  70%  in  19922; 
Cost  Savings 

o   Reported  charges  for  births  of 

HealthStart  newborn  infants  ($1,739) 
were  significantly  lower  than  those 
of  other  Medicaid  newborn  infants 

($2,068)1; 
o   The  cost  savings  estimate  for  the 

decrease  in  the  percentage  of  newborn 
infants  admitted  to  the  NICU  in  1990 
based  on  the  reported  charge  of 
$8,300  per  infant  was  $4.1  million  in 
1990,  nearly  the  entire  cost  to  NJ 
for  the  HealthStart  services  provided 

in  that  year  '  . 
Other  Studies  Using  the  HealthStart  Data 

Confirmation  of  NJ  HealthStart 
Evaluation: 


Table  1:    Logistic  Regression  and 
Adjusted  Odds  Ratios  for 
HealthStart  Maternal 
Characteristics   and  Low 
Birth  Weight  Newborns 


In  addition  to  the  evaluation  of 
the  HealthStart  program  conducted  by 
NJDOH  and  NJDHS,  a  collaborative 
evaluation  of  the  HealthStart  program 
was  conducted  with  an  independent 
evaluator,  an  economic  and  population 
researcher.   Using  the  HealthStart 
Linked  Data  System,  and  ordinary  and 
two-stage  least  sguares  multivariate 
analysis  of  variance  as  the  primary 
statistical  procedure,  this  study 
confirmed  the  results  of  the  NJ 
evaluation  findings  that  HealthStart  has 
had  an  impact  on  maternal  and  infant 

4 
health  outcomes  and  costs  .   This  study 
also  confirmed  that  self-selection  bias 
was  not  a  limiting  factor  of  the 
findings.   HealthStart  women  were  found 
to  be  nearly  identical  to  other  Medicaid 
women  in  age,  education,  race,  and 
marital  status;  and  to  be  even  more  at 
risk  of  adverse  outcomes  than  other 
Medicaid  pregnant  women. 
Factors  Related  to  Low  Birthweight: 
A  multiple  logistic  regression  was 
performed  using  SAS  LOGIST  to  determine 
the  relationship  between  low  birth 
weight  (less  than  2500  grams)  and  three 
sets  of  variables  in  the  MSSD  data:  the 
HealthStart  woman's  demographic  and 

behavioral  characteristics,  and  her 

5 
service  utilization  .   The  Odds  Ratios 
based  on  the  multiple  logistic  regres- 
sion, adjusted  for  all  other  variables 
in  the   model,  showed  that  women  younger 
than  20,  older  than  34;  unmarried;  or 
black;  and  those  who  smoked,  drank,  or 
used  drugs;  had  a  significantly  higher 
risk  of  a  low  birth  weight  baby.   Women 
who  received  future  family  planning, 
childbirth  education,  or  WIC  (three  of 
the  tenets  of  services  provided  by  the 
HealthStart  program, )  or  had  a  male 
baby,  had  a  significantly  lower  risk  of 
a  low  birth  weight  baby  (Table  1). 


CHARAC . 

ADJ. 

CHARAC . 

ADJ. 

ODDS 

ODDS 

RATIO 

RATIO 

AGE 

RISKS 

<20 

1.3* 

Smoker 

1.6* 

20-34 

1 

Not  Smoker 

1.0 

35+ 

1.7* 

Drinker 

1.4* 

MARITAL 

Not  Drinker 

1.0 

STAT. 

Drug  User 

1.8* 

Married 

1.0 

Not  User 

1.0 

Not  Marr. 

1.2* 

SERVICE 

RACE/ETH 

Family  Plan. 

.8* 

White 

1.0 

No  Fam.Plan. 

1.0 

Black 

1.8* 

Childbir.Ed. 

.  9* 

Hispanic 

1.1 

No  Child. Ed. 

1.0 

Other 

1.6* 

WIC 

.  9* 

SEX  OF  BABY 

No  WIC 

1.0 

Male 

.8* 

Female 

1.0 

*   p<=.05 

,  95%  C. 

I.  does  not  include 

Transfer  of  HealthStart  Model  of 
Prenatal  Care  and  Application  of 
HealthStart  Evaluation  Methodology  to 
Managed  Care 

The  HealthStart  model  of  prenatal 
care  has  been  extended  in  many  hospital- 
based  agencies  to  all  women  who  come  for 
prenatal  care.   In  1992  and  1993  the 
model  was  expanded  to  include  the  non- 
Medicaid  population  up  to  300%  of  the 
Federal  poverty  through  a  joint 
collaboration  of  the  NJDOH,  NJDHS  and 
U.S.  Healthcare. 

NJ  is  now  in  the  process  of 
adapting  the  program  and  the  evaluation 
methodology  as  we  implement  a  Medicaid 
managed  care  system  for  the  AFDC 
population.   A  comparable  methodology  to 
that  used  in  the  evaluation  of  the 
current  HealthStart  program  is  being 
used  to  evaluate  the  HealthStart  program 
under  the  managed  care  system. 
Notes : 

1.  HealthStart  Linked  Data  System 
(which  excluded  multiple  births  and 
out-of-NJ  hospital  births). 

2.  HealthStart  MSSD  data  (which 
includes  all  births;   the 
percentages  are  somewhat  different, 
therefore,  in  this  data  set  than  in 
the  Linked  Data  System). 

3.  HealthStart  Pediatric  Preventive 
Health  Care  Visit  Form. 

4.  Nancy  Reichman  and  Maryanne  Florio. 
"The  Effect  of  Enriched  Prenatal 
Care  Services  on  Medicaid  Birth 
Outcomes  in  New  Jersey"   Princeton 
University  Office  of  Population 
Research  Working  Paper  No.  95-2: 
Princeton,  New  Jersey.  January  1995. 

5.  Lucille  Wallington.  "A  Study  of  the 
Determinants  of  Low  Birthweight 
Among  HealthStart  Program  Partici- 
pants in  New  Jersey"  Unpublished 
paper  to  fulfill  the  requirements 
for  the  Columbia  University  Enhanced 
Analytical  Skills  Program  in  cooper- 
ation with  New  Jersey  Department  of 
Health.  May  1993. 
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MOTHERS  AND  INFANTS 
Discussant 

Woodie  Kessel,  Maternal  and  Child  Health  Bureau 


Comments  not  available  for  publication. 
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Cognitive  impairment 

of  the  Elderly 


MEASUREMENT  OF  COGNITION  IN  AHEAD: 
METHODOLOGICAL  AND  SUBSTANTIVE  INVESTIGATIONS 

A.  Regula  Herzog,  University  of  Michigan 
Robert  B.  Wallace,  University  of  Iowa 


The  Asset  and  Health  Dynamics  Among  the 
Oldest  Old  (AHEAD)  study  is  a  biennial  survey  of 
a  nationally  representative  sample  of  more  than 
8,000  community-residing  persons  70  years  old 
and  older  in  1993/94.   The  survey  focuses  on 
health  transitions  in  late  life  and  their 
economic  and  family  consequences.   Because 
cognitive  changes  are  an  important  component  of 
such  transitions,  an  effort  was  made  to  properly 
measure  cognition.   A  broadly  differentiating 
measure  was  sought  to  investigate  change  in 
cognitively  well  functioning  as  well  as  in 
already  impaired  persons.   Persons  between  70 
and  80  were  interviewed  primarily  on  the 
telephone,  persons  over  80  face-to-face. 

The  dimensions  of  memory  for  new 
information  and  working  memory  --or  the  ability 
to  keep  information  in  store  while  processing  it 
--  were  considered  important  for  the  measurement 
of  cognition  in  the  AHEAD  because  they  are  known 
to  decline  throughout  a  good  part  of  the  adult 
life  span.   The  dimension  of  established 
knowledge  was  considered  important  because  it  is 
thought  to  decline  less  substantially  and  at  a 
later  point  in  the  life  span  than  memory. 

These  dimensions  of  cognition  were 
represented  by  the  following  measures.   Memory 
for  new  information  was  measured  by  an  immediate 
and  a  delayed  free  recall  test  of  10  nouns. 
Working  memory  was  measured  by  Serial  7s  (i.e., 
5  consecutive   subtractions  of  7  from  100) . 
Established  knowledge  was  measured  by  vocabulary 
items  and  by  questions  on  orientation  in  time, 
space,  and  history.   These  measures  were  adapted 
from  the  Telephone  Interview  for  Cognitive 
Status  (TICS)  developed  by  Brandt  and  colleagues 
(1988)  . 

In  psychometric  terms,  the  different 
measures  interrelate  moderately  (average  r  = 
.24),  indicating  a  common  underlying  construct 
of  cognition.   The  different  dimensions  were 
probed  with  factor  analytic  techniques.   An 
exploratory  factor  analysis  suggests  a  memory 
factor  and  a  factor  including  all  other 
measures.   A  confirmatory  factor  analysis  that 
models  a  working  memory  factor  in  addition  to 
the  memory  and  "knowledge"  factors  fits  the  data 
well,  potentially  supporting  a  separate  working 
memory  component . 

These  cognitive  measures  differ 
considerably  in  difficulty  level  (Table  1) .   The 
delayed  recall  measure  is  most  difficult,  the 
knowledge  measures  are  easiest,  the  Serial  7s 
and  immediate  recall  measures  are  of 
intermediate  difficulty.   Furthermore,  the 
measures  do  provide  the  desired  differentiation 
at  higher  as  well  as  lower  levels  of  cognitive 
functioning.   For  example,  about  one  quarter  of 
respondents  who  did  poorly  on  the  delayed  recall 
measure  could  still  answer  correctly  all 
knowledge  questions.   Or,  about  one  quarter  of 
respondents  who  correctly  answered  all  knowledge 
questions,  could  recall  only  2  or  fewer  words  in 
the  delayed  condition. 

Missing  data  are  not  always  negligible; 
about  3,  3,  and  12  percent  of  the  respondents 
refused  to  answer  to  the  immediate  memory, 
delayed  memory,  and  Serial  7s  tasks, 
respectively.   Fewer  refused  the  knowledge 
questions.   Several  observations  suggest  that 
respondents  who  refused  did  not  know  the  answer: 
(1)  refusals  are  higher  on  difficult  than  easy 
measures;  (2)  refusals  increase  with  age;  (3) 
those  who  refused  Serial  7s  did  no  better  on  the 
knowledge  measures  then  those  who  did  not  refuse 
but  obtained  a  low  score  and  they  described 


their  cognitive  functioning  as  no  better;  (4) 
those  who  refused  looked  similar  on  several 
health  measures  to  those  who  did  not  refuse  but 
obtained  low  scores.   These  findings  suggest 
that  incorrect  answers  can  be  imputed  for 
refusals,  a  conclusion  also  reached  by  other 
investigators . 

A  total  cognitive  score  was  formed  by 
summing  the  single  measures  after  imputing 
incorrect  scores  for  refused  answers .   This 
measure  yields  an  approximately  normal 
distribution,  which  is  quite  different  from  the 
skewed  distributions  obtained  with  typical 
geriatric  mental  status  scales  in  community 
populations . 

An  important  methodological  issue  facing 
the  AHEAD  design  was  whether  the  mode  of 
interviewing  would  affect  the  performance  on  the 
cognitive  measures.   It  seemed  possible  that  a 
cognitive  test  would  be  more  difficult  to 
administer  and  to  respond  to  over  the  telephone 
than  face-to-face.   Because  in  the  AHEAD  design 
mode  was  confounded  with  age  and  thus  presumably 
with  capacity,  adjustment  for  these  differences 
was  necessary.   OLS  regression  analyses  of 
cognitive  functioning  on  mode  with  and  without 
controls  for  initial  differences  showed  an 
uncontrolled  difference  of  three  points  on  the 
aggregate  measure  to  shrink  to  a  difference  of 
less  than  one  half  point  with  controls.   This 
finding  suggests  that  most  of  the  mode 
difference  was  compositional  and  not  due  to  mode 
of  administration  per  se . 

Another  important  methodological  issue 
facing  AHEAD  is  whether  being  present  at  a 
household  member's  interview  improves 
performance  on  the  cognitive  measures  for  the 
second  household  member  to  be  interviewed.   A 
difference  of  more  than  one  point  on  the 
aggregate  measure  was  found,  even  after  a  number 
of  differences  between  household  members  had 
been  controlled,  suggesting  that  there  is  a 
learning  effect  associated  with  listening  in  on 
the  test  administration  to  a  different  person 
(Rodgers,  unpublished  tabulations) . 

A  final  issue  is  whether  the  cognitive 
measures  in  AHEAD  show  construct  validity  by 
replicating  established  findings  and  by  behaving 
in  theoretically  predicted  ways.   An  OLS 
regression  analysis  shows  the  following 
replicated  findings.   Adults  who  are  old,  have 
little  formal  education  and  are  in  relatively 
bad  physical  health  perform  relatively  badly  on 
the  aggregate  cognitive  measure.   Less  well 
established  but  theoretically  sensible  findings 
include  positive  effects  of  hearing  and  vision, 
of  high  income  and  assets,  of  moderate  drinking, 
and  of  a  lack  of  difficulties  with  activities  of 
daily  living  on  cognitive  performance. 

In  summary,  the  AHEAD  experience 
demonstrates  that  cognitive  performance  measures 
can  be  administered  by  lay  interviewers  in  large 
scale  survey  efforts.   Despite  limited  time,  a 
broadly  differentiating  measure  of  promising 
validity  was  obtained.   Aspects  of  surveys  such 
as  missing  data  and  mixed  modes  did  not 
represent  unsurmountable  problems .   Whereas 
other  aspects  such  as  the  effect  of  educational 
attainment  and  of  interviewing  respondents  in 
the  same  household  will  need  further  attention, 
the  cognitive  measures  developed  for  AHEAD  have 
positioned  it  well  for  investigating  the  effect 
of  cognitive  functioning  and  changes  therein  on 
economic  and  family  resource  use. 
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TABLE  1 
Difficulty  Levels  of  Cognitive  Tests 


Immediate  recall: 

Mean  #  of  10  nouns 

Delayed  recall: 

Mean  #  of  10  nouns 

Serial  7s: 

Mean  #  of  5  subtractions 

Knowledge  items : 

Mean  #  of  4  dates 

%  counting  backwards  correctly 

%  naming  scissors  correctly 

%  naming  cactus  correctly 

%  naming  president  correctly 

%  naming  vice-president  correctly 


4.4 

2.9 

3.1 

3.6 

90% 
98% 
83% 
89% 
72% 
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COMMUNITY  SCREENING  FOR  DEMENTIA:  THE  MINI  MENTAL  STATE  EXAM  (MMSE) 
AND  MODIFIED  MINI-MENTAL  STATE  EXAM  (3MS)  COMPARED 


I.  McDowell1,   E.  Kristjansson1,   G.B.  Hill1,   R.  Hebert1' 
1 .  University  of  Ottawa    2.  Hopital  de  Sherbrooke 


The  Mini-Mental  State  Examination 
(MMSE)  was  developed  as  a  bedside  tool 
to  evaluate  the  cognitive  status  of 
elderly  people  in  clinical  settings 
(1)  ;  it  is  also  used  as  a  survey- 
instrument  to  screen  for  cognitive 
impairment  and  dementia  (2,3) .   It  is 
brief  and  easy  to  administer,  and  has 
shown  good  reliability  (2) .   Validity 
as  a  screening  test  is  generally 
acceptable  (3) ,  although  certain 
limitations  have  been  identified,  for 
example  with  psychiatric  patients  (4)  , 
and  in  identifying  focal  brain 
dysfunction  (5)  or  mild  dementia  (3) . 
On  a  practical  level,  the  MMSE  lacks 
an  administration  manual,  so  that 
scoring  and  interpretation  seem  to 
vary  between  users  (6) . 

In  1987  Evelyn  Teng  proposed  the 
Modified  Mini-Mental  State  Exam  (MMMS, 
or  3MS)  to  respond  to  these  issues 
(7) .   She  added  four  items  to  cover  a 
broader  range  of  cognitive  functions 
and  difficulty  levels.   The  complete 
3MS  covers  orientation  to  time  and 
place,  registration,  recall,  simple 
language,  and  construction.   Teng  also 
increased  the  scoring  range  from  3  0  to 
100  points  to  provide  finer 
discrimination,  and  she  produced  a 
detailed  manual  to  standardize 
administration.   Preliminary  validity 
and  reliability  results  for  the  3MS 
were  good:  test-retest  reliability 
ranging  from  0.91  to  0.93,  and  a 
sensitivity  of  91%  for  dementia,  at  a 
specificity  of  97%  (7) .   As  most  items 
in  the  MMSE  and  3MS  overlap,  an  MMSE 
score  can  be  derived  from  the  3MS  with 
the  addition  of  a  few  extra  questions. 

The  3MS  requires  slightly  longer 
to  administer  and  to  score  than  the 
MMSE  and  so  is  worthwhile  only  if  it 
offers  significantly  superior 
validity.   We  address  two  questions: 

1.  Is  the  validity  of  the  3MS 
superior  to  that  of  the  MMSE  in  a 
community  sample? 

2.  If  so,  is  the  improvement  due 
to  the  broader  scope  of  the  3MS, 
or  to  its  more  detailed  scoring 
system,  or  to  both? 


The  comparison  in  a  community 
sample  is  relevant  because  screening 
tests  often  do  not  perform  as  well  in 
general  population  samples  as  they  do 
in  more  highly  selected  clinical 
samples  (8) .   The  severity  of  dementia 
is  generally  higher  in  clinical 
referral  samples  than  in  unselected 
population  samples,  which  inflates  the 
estimate  of  sensitivity  compared  to 
that  obtained  in  a  population  sample 
(9) .   Validation  in  a  community  study, 
however,  must  address  methodological 
problems  such  as  verification  bias. 
This  occurs  where  people  are  selected 
to  receive  the  gold  standard  clinical 
examination  on  the  basis  of  their 
scores  on  the  screening  test. 
Typically  it  is  not  feasible  to 
clinically  examine  everyone,  and  so 
all  the  subjects  who  screen  positive, 
but  only  a  sub- set  of  the  majority  who 
screen  negative,  are  referred  for 
diagnostic  work-up.   The  resulting 
bias  inflates  estimates  of  sensitivity 
and  diminishes  specificity,-  methods 
have  been  described  for  its  correction 
(10-13) .   The  bias  also  affects  the 
area  under  the  receiver  operating 
characteristic  curve  (AUC) ,  an 
indicator  of  validity  (13). 


Methods 

The  data  are  taken  from  the  Canadian 
Study  of  Health  and  Aging  (CSHA) ,  a 
multi- centre  study  of  the  epidemiology 
of  dementia,  health  and  disability 
among  Canadians  aged  65  and  over  (14)  . 
Eighteen  study  centres  were  involved; 
in  the  community,  an  age-stratified 
random  sample  (N  =  9,008)  was  drawn; 
the  institutional  sample  (N  =  1,250) 
is  not  considered  in  the  present 
report . 

In  the  community  sample,  the  3MS 
was  administered  as  a  cognitive 
screening  test  in  a  home  interview,  in 
English  or  French.   Interviewers 
received  a  five-day  training  session 
on  administering  and  scoring  the 
screening  instrument.   The  French 
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version  of  the  3MS  had  been  tested 
previously,  giving  intraclass 
correlations  of  0.94  for  test-retest 
reliability  and  0.95  for  inter-rater 
reliability.   Internal  consistency  was 
also  high  (alpha  =  0.89,  split-half  = 
0.93)  (15) .   Based  on  a  pilot  study 
(16),  a  cutting  point  of  77/78  was 
chosen  as  optimal,  paying  especial 
attention  to  maintaining  high 
sensitivity. 

Of  the  9,008  community  subjects, 
8,949  completed  the  screening 
interview  and  2,398  were  referred  for 
clinical  assessment.   These  included 
1,673  who  screened  positive  on  the  3MS 
and  725  who  screened  negative.   Of  the 
2,398,  39  (1.6%)  were  found  not 
eligible,  105  (4.5%)  were  not 
accessible,  and  559  (26.4%)  refused  to 
attend  the  clinical  examination.   This 
resulted  in  1600  clinical  evaluations: 
494  on  people  who  screened  negative 
and  1106  on  those  who  screened 
positive . 

A  clinical  examination  verified 
the  presence  of  cognitive  impairment, 
and  provided  a  differential  diagnosis 
of  dementia.   Diagnostic  criteria  were 
based  on  the  DSM-III(R)  (17)  and  on 
the  ICD-10  (18) .   The  clinical 
examination  involved  a  medical  and 
family  history;  a  mental  status 
assessment,  physical  and  neurological 
examination  by  a  physician;  and  13 
neuropsychological  tests.   Consensus 
diagnoses,  involving  physician  and 
neuropsychologist,  were  made  blind  to 
the  3MS  scores.   They  classified  the 
person  as  cognitively  normal  (N  = 
747) ,  cognitively  impaired  but  not 
demented  (CIND,  N  =  488) ,  or  demented 
(N  =  365) .   Dementia  was  classified 
into  probable  Alzheimer's  disease, 
possible  Alzheimer's  disease  (with  or 
without  a  vascular  component)  multi- 
infarct  dementia  and  unclassif iable 
dementia.   The  CIND  diagnosis 
represents  an  attempt  to  classify 
people  with  recognizable  cognitive 
decline  that  did  not  currently  meet 
the  criteria  for  dementia.   This  group 
was  sub-divided  into  age-associated 
memory  impairment,  impairments  due  to 
psychiatric  illnesses,  vascular 
problems  and  apparent  very  early 
stages  of  dementia.   In  this  paper  we 
do  not  distinguish  between  these  sub- 
categories . 

Validity  analyses  used  the 
receiver  operating  curve  (ROC) 
approach,  comparing  the  screening 
tests  graphically  and  statistically 
across  a  range  of  possible  cutting 


points  (19) .   The  area  under  the  ROC 
curve  (AUC)  indicates  the  amount  of 
information  provided  by  the  test :  a 
test  performing  no  better  than  chance 
would  have  an  AUC  of  0.5  and  a  perfect 
test  would  have  an  AUC  of  1.0  (19)  . 
All  validity  analyses  were  corrected 
for  verification  bias.   A  program  was 
written  in  Basic  for  this,  following 
the  method  outlined  by  Gray  and  Begg 
(10) .   This  uses  the  distribution  of 
3MS  scores  for  the  screening  sample 
and  the  diagnostic  status  of  those 
clinically  examined  at  each  3MS  score 
as  the  basis  for  calculating  corrected 
sensitivity  and  specificity  for  each 
score . 

The  AUC  indicates  the  overall 
performance  of  a  screening  test 
across  all  possible  cutting  points;  in 
practice,  however,  one  uses  a  single 
cutting  point.   The  choice  of  a 
suitable  cutting  point  should  consider 
the  relative  importance  of  false 
positive  and  false  negative  errors. 
This  can  be  formalized  by  assigning  a 
weight  to  represent  the  relative 
importance  of  avoiding  each  type  of 
error,  and  an  optimal  cut-point  can 
then  be  chosen  for  each  test .   This  is 
often  done  implicitly;  one  way  to  make 
the  process  explicit  is  through  a 
decision  theory  approach  in  which  the 
chosen  cut-point  is  that  which  equates 
the  ratio  of  the  posterior 
probabilities  of  false  positive  and 
false  negative  errors  with  the 
reciprocal  of  the  ratio  of  their 
costs,  or  dysutilities.   In  comparing 
two  screening  tests,  the  one  that 
produces  the  lower  expected  loss  is  to 
be  preferred.   We  made  this  comparison 
for  the  3MS  and  MMSE,  varying  the  cost 
ratios  from  1:10  to  10:1,  and 
identifying  the  cut -point  that 
produced  the  smallest  expected  loss 
for  each  test  at  each  cost  ratio. 


Results 


The  3MS  was  acceptable  to  most 
subjects.   Fifty-nine  out  of  9,008 
were  unable  to  complete  the  3MS  test; 
upon  clinical  examination,  63%  of 
these  were  demented  and  22%  were 
cognitively  impaired. 

A  comparison  of  the  English-  and 
French- language  samples  (N  =  1166  and 
434,  respectively)  indicated  that  they 
were  comparable  in  age  and  gender,  but 
that  the  French  language  sample  had 
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lower  educational  levels  (an  average 
of  6.8,  compared  to  9.2  years;  F  = 
114.7,  df  =  1,  p  =  .0001;  Table  1). 


Table  1.   Validation  Sample  Description 

English     French      Total 
mean     (n=1166)     (n=434)     (n=1600) 


Age                 80.3 

79.1 

80.0 

Education         9.2 

6.8 

8.6 

3MS  score       73.5 

71.3 

72.8 

MMSE  score    23.1 

23.0 

23.1 

Validity  of  MMSE  and  3MS 

Dementia  versus  non-dementia.   At  the 
cut-off  of  77/78  chosen  for  our  study 
and  correcting  for  verification  bias, 
the  3MS  had  87%  sensitivity  for 
detecting  dementia  and  a  specificity 
of  89%  in  the  English-language  sample 
The  ROC  curves  are  shown  in  Figure  1 
and  suggest  that  the  performance  of 
the  3MS  was  superior  to  the  MMSE  at 
almost  all  cutting  points.   The  area 
under  the  curve  for  the  3MS  was  0.94, 
and  that  for  the  MMSE  was  0.89  (Z  = 
5 .38,  p  <  .01) . 


There  were  also  significant 
differences  in  diagnosis  between  the 
English  and  French  samples  (chi-square 
=6.2,  df  =  2,  p  =  .04;  Table  2) . 
Accordingly  we  analyze  the  performance 
of  the  screening  tests  separately  in 
the  two  groups . 

Table  2.   Validation  Sample:  Diagnosis 


Sensitivity 


English 

French 

Total 

(n=1166) 

(n=434) 

(n=1600) 

diagnosis 

Normal           45% 

51% 

47% 

CIND               31% 

30% 

30% 

Demented      24% 

19% 

23% 

1 

~    <• 

0.8 

"     /      / 

0.6 

/    / 

i : 

0.4 

4/ 

r:         / 

3  MS 

0.2 

ft-                                                                         J0^ 

■  ■  ■  ■  MMSE 

n 

'    1        1        1 

No  Information 

i       i       i       i       i       i 

0      0.1    0.2   0.3   0.4   0.5   0.6   0.7   0.8    0.9     1 

1  -  Specificity 

Figure  1 .   ROC  Curves  for  3MS  and  MMSE 
for  Dementia 


Internal  consistency.   The  alpha 
internal  consistency  coefficient  for 
the  3MS  (English  sample)  was  0.87; 
split-half  reliability  was  0.82.   The 
consistency  of  the  MMSE  was  lower: 
coefficient  alpha  was  0.78  and  split- 
half  reliability  was  0.76.   The 
contrast,  however,  is  largely  due  to 
the  greater  number  of  items  on  the 
3MS;  application  of  the  Spearman-Brown 
formula  to  correct  for  the  different 
lengths  of  the  two  tests  (20)  shows 
that  the  MMSE  would  have  a  coefficient 
alpha  of  0.86  if  it  had  the  same 
number  of  items  as  the  3MS .   Both 
scales  had  lower  internal  consistency 
in  the  French- language  sample:  a   = 
0.82  for  the  3MS  and  a   =    0.74  for  the 
MMSE;  the  split-half  figures  were  0.79 
and  0.73,  respectively. 


To  illustrate  whether  the  difference 
between  the  two  ROC  curves  in 
identifying  dementia  holds  clinical 
implications,  Figure  2  presents  the 
specificity  of  the  two  tests  at 
comparable  sensitivity  levels 
(English- language  sample) .   At 
sensitivity  levels  up  to  0.93,  the 
MMSE  would  falsely  classify  as 
demented  about  7%  more  people  than 
would  the  3MS .   At  higher 
sensitivities,  the  advantage  of  the 
3MS  increases . 

This  contrast  is  further 
illustrated  by  comparing  the  minimum 
expected  losses  produced  by  the  tests 
for  particular  cutting  points 
reflecting  various  weightings  of  false 
positives  versus  false  negatives 
(Table  3) .   The  3MS  is  always  slightly 
superior  to  the  MMSE;  the  contrast 
becomes  stronger  as  sensitivity  is 
weighted  more  heavily.   For  example, 
when  false  negatives  are  weighted  as 
10  times  more  important  then  false 
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Table  3:  Comparison  of  Minimum  Expected  Loss  of  MMSE  and  3MS  Screening  Tests  in  identifying 
Dementia,  at  various  ratios  of  the  cost  of  False  Negatives  to  that  of  False  Positives. 


MMSE 

SMS 

Loss  Ratio 

Optimal  Cut- 

Sens., 

Minimum 

Optimal  Cut- 

Sens.. 

Min. 

(False  Neg/ 

Point 

Spec. 

Expected  Loss 

Point 

Spec. 

Expec 

False  Pos) 

Loss 

1:10 

8 

3,  99.9 

.088 

31 

6,  99.9 

.08 

1:3 

13 

8,  99.5 

.083 

42 

13,  99.9 

.063 

1:2 

15 

16,  99.5 

.082 

50 

22,  99.6 

.06 

1:1 

16 

20,99 

.076 

61 

41,  99 

.053 

2:1 

20 

31,  98 

.14 

67 

56,  97 

.087 

3:1 

22 

44,  95 

.186 

69 

61,  96 

.116 

6:1 

24 

63,  89 

.27 

74 

76,  92 

.169 

10:1 

25 

86,  77 

.33 

77 

86,  87 

.218 

positives,  the  optimal  cut-point  for 
the  3MS  is  77  and  the  expected  loss 
for  the  3MS  was  0.22  compared  to  0.33 
for  the  MMSE  at  its  optimum  cut -point 
of  25  . 


Specificity 


Sensitivity 


0.8 
0.6 
0.4 
0.2 


7.7 

.  ^ 


3  MS 

■  ■  •  ■ MMSE 

J I i_ 


0.62    0.67    0.72    0.77    0.82    0.87    0.92    0.97 
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Figure  2 .   Comparison  of  Specificity  at 
Various  Sensitivity  Levels 

Dementia  +  Cognitive  Impairment  versus 
Normal .   If  the  task  is  to  distinguish 
all  levels  of  cognitive  impairment 
(i.e.,  CIND  plus  dementia)  from 
cognitive  normality,  neither  test 
performed  well .   The  performance  of 
the  3MS  was  very  slightly  superior  to 
that  of  the  MMSE;  the  3MS  had  an  AUC 
of  0.80,  compared  to  0.78  for  the  MMSE 
(p  <  0.05)  (see  Figure  3).   The 
minimum  expected  losses  calculations 
again  showed  the  3MS  to  be  slightly 
superior . 


0    0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 

1  -  Specificity 

Figure  3 .   ROC  Curves  for  3MS  and  MMSE 
in  Screening  for  Cognitive 
Impairment  and  Dementia 


Extended  Length  versus  Scoring.   Was 
the  superiority  of  the  3MS  due  to  its 
greater  length,  or  to  the  altered 
scoring  system?   If  the  relative 
advantage  of  the  3MS  were  due  to  its 
wider  range  of  scoring  alone,  then 
there  should  be  no  difference  in 
validity  results  for  the  two  tests  if 
the  3MS  questions  were  re-scored  to 
conform  to  the  approach  used  in  the 
MMSE.   Retaining  the  added  items,  but 
re-scoring  the  3MS  using  the  approach 
of  the  MMSE,  yielded  an  AUC  of  0.90, 
only  marginally  higher  than  the  0.89 
of  the  MMSE,  and  lower  than  the  0.94 
for  the  3MS  as  correctly  scored. 
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If  the  advantage  of  the  3MS  were 
attributable  to  the  four  extra 
questions,  there  should  be  no 
difference  in  validity  of  the  MMSE  and 
the  3MS  if  scores  for  the  3MS  were  re- 
calculated omitting  the  four  new  items 
but  using  the  3MS  scoring  method  for 
the  remaining  items.   The  AUC  for 
dementia  for  the  3MS  without  the  four 
questions  was  0.91,  lower  than  for  the 
complete  3MS  (0.94),  but  higher  than 
the  0.89  for  the  MMSE.   These  results 
suggest  that  the  superiority  of  the 
3MS  is  due  to  both  changes,  with  very 
slightly  more  of  the  effect  being  due 
to  the  changes  to  the  scoring 
procedure . 


Discussion 


The  present  sample  is  large  and 
representative  of  the  type  of  elderly 
person  likely  to  consult  with  primary 
care  physicians.   Our  "gold  standard" 
diagnosis  of  dementia  followed 
established  criteria  and  used  a 
thorough  examination  that  included 
neuropsychological  testing.   Inter- 
rater  agreement  on  diagnosis  was  high, 
comparing  well  with  findings  of  other 
studies.   Generalizability  was 
enhanced  by  comparing  the  tests  in  two 
language  groups.   The  correction  for 
verification  bias  provides  valid 
figures  for  sensitivity,  specificity 
and  loss  functions. 

Overall,  Teng's  3MS  does  improve 
on  the  MMSE,  although  there  is  still 
room  for  improvement,  especially  in 
the  identification  of  early  dementias. 


The  study  suggests  that  the  extra  time 
and  cost  of  using  the  3MS  translates 
into  more  accurate  screening  results 
than  those  achieved  using  the  MMSE. 
The  contrast  is  illustrated  when  the 
results  are  mapped  back  onto  the  full 
community  sample  of  8,949  people.   For 
example,  at  a  loss  ratio  of  2:1,  the 
3MS  would  miss  an  estimated  228  cases 
of  dementia  and  would  falsely  classify 
an  estimated  392  normal  people  as 
demented.   At  the  same  loss  ratio,  the 
MMSE  would  have  missed  an  estimated 
491  cases  of  dementia  at  the  cost  of 
263  false  positives.   Changing  the 
false  negative  to  false  positive  loss 
ratio  to  10:1,  the  3MS  would  miss  an 
estimated  85  cases  of  dementia  in  the 
sample  and  an  estimated  1098  normal 
people  would  be  referred  unnecessarily 
for  clinical  examinations.   The  MMSE 
would  miss  an  estimated  106  cases  of 
dementia  and  would  have  entailed  1864 
unnecessary  clinical  examinations. 

The  increased  accuracy  of  the  3MS 
appears  due  both  to  the  additional 
questions  and  expanded  scoring.   The 
expanded  scoring  increases  test 
variance  which  results  in  improved 
validity,  other  things  being  equal 
(20) .   It  also  allows  for  finer 
discrimination.   It  is  reasonable  that 
the  additional  items  should  enhance 
the  validity  of  the  test:  they  assess 
recall,  which  is  one  of  the  most 
significant  impairments  in  cognitive 
impairment  and  dementia,  long-term 
memory  which  is  a  criterion  for 
dementia  in  the  DSM-III-R,  and  verbal 
fluency  and  abstraction,  which  are 
commonly  impaired  in  dementia. 
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Introduction 

Deficits  in  cognitive 
functioning  are  prevalent  among 
the  elderly.  Recent  studies  of 
noninstitutional ized 
populations  indicate  that  20%, 
or  more,  of  the  elderly  show 
signs  of  cognitive  impairment. 
Among  populations  under 
treatment  in  mental  health 
clinics,  cognitive  impairment 
has  been  found  to  be  a  risk 
factor  for  shortened  survival 
and  early  institutionalization 
as  well  as  for  dementia. 

There  have  been  few 
studies  which  have  investigated 
the  effects  of  cognitive 
impairment  on  the  daily  lives 
and  activities  of  elderly  men 
and  women  residing  in  the 
community.  Although  the  impact 
on  caregivers  of  relatives  with 
cognitive  impairment  has  been 
studied.  Perhaps  because  the 
effects  of  cognitive  impairment 
on  the  quality  of  life  of 
clinically  diagnosed  demented 
individuals  seemed  so  obvious, 
further  investigation  among 
non-demented  impaired 
individuals  may  have  appeared 
to  be  unnecessary. 

Within  recent  years, 
interest  in  "Quality  of  Life" 
(QL)  determination  and 
measurement  has  grown  as  a  way 
to  assess  intervention  efforts, 
and/or  change  in  population 
health  status  over  time 
especially  with  regard  to 
individuals  with  chronic 
conditions.  Consensus  on  QL 
definitions  and  measurement, 
however,  continues  to  elude 
interested  investigators. 
Conceptual  (and  measurement) 
schemes  range  from 
determination  of  a  limited  set 
of  health  status  variables 
embodied  under  the  rubic  of 
Health  Related  Quality  of  Life 
to  conceptual  schemes 
incompassing  15  dimensions  and 


over  1,000  empirical  measures. 
Clearly,  when  individuals  are 
sick  or  chronically  ill  or 
impaired,  all  aspects  of  life 
may  be  affected  beyond  those 
traditionally  defined  as 
"Health  Status"  or  physical 
functioning.  In  this 
presentation  data  on  the 
quality  of  life  of  community 
residing  elderly  persons  are 
related  to  varying  levels  of 
cognitive  functioning.  The 
domains  and  measures  of  QL  are 
shown  below.  (See  Figure  1) 


Figure  1 


Health 


-Medical  Conditions 
-Sleep  Difficulties 
-Subjective  Health 
-Symptoms  of  Depression 
-Behavioral  Problems 
-Mood  (positive  affect) 

B.  Functioning 

-Social  Support 

-Problems  with  Daily  Activities 

-Nursing  Home  Residence 

C.  Personal  Relationships 

-Number  of  Friends 
-Contact  with  Children 
-Contact  with  Others 
-Intimacy 

D.  Activities  outside  the  home 

-Gets  out  of  the  house 
-Religious  Attendance 
-Organizational  Participation 
-Working 
-Exercise 

Sample 

Data  for  this  analysis  are 
taken  from  baseline  personal 
interviews  with  1855  community 
residents  at  least  65  years  of 
age  who  were  participants  in  a 
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longitudinal  study  of  aging  and 
health  (Norwood-Montef iore 
Aging  Study  [NMAS])  .  The  sample 
was  randomly  selected  from  a 
list  of  Medicare  beneficiaries 
living  in  a  neighborhood  in  the 
Bronx  in  New  York  City.  Two 
thirds  of  the  sample  were 
women,  more  than  half  were  7  5 
years  of  age  or  older,  41%  were 
married,  and  44%  lived  alone. 
The  mean  annual  income  in  1984 
was  $12,039.  Two  thirds  had 
private  health  insurance  in 
addition  to  Medicare,  and 
nearly  15%  received  Medicaid. 
The  small  proportions  of  non- 
Whites  and  Hispanics  reflect 
the  pattern  of  housing 
segregation  in  the  study  area. 
Baseline  interviews  were 
conducted  between  July  1984  and 
March  1985  and  were  the  first 
in  a  series  of  semi-annual 
contacts  over  3  1/2  years.  The 
response  rate  to  the  baseline 
interview  was  73%.  Ninety-five 
percent  or  more  of  baseline 
respondents  were  reinterviewed 
in  each  successive  interview 
wave. 

Measures 

The  Mini-Mental  State  Exam 
(MMSE) ,  an  instrument  widely 
used  to  screen  for  cognitive 
impairment,  was  administered  to 
all  study  participants  at 
baseline.  Developed  by  Folstein 
et  al.,  the  MMSE  assesses  signs 
of  dysfunction  in  orientation, 
registration,  attention, 
calculation,  recall,  and 
language. 

MMSE  scores  range  from  0 
to  30.  Persons  with  scores  of 
less  than  18  are  defined  as 
severely  impaired,  those  with 
scores  of  18  to  23  as  mildly 
impaired,  and  those  with  scores 
of  24  and  above  as  unimpaired. 
Although  the  MMSE  is  an  easily 
administered  and  reliable  tool 
to  identify  signs  of  cognitive 


impairment,  scores  do  not 
provide  clinical  diagnoses. 
Cognitive  impairment  may  result 
from  a  variety  of  mental 
conditions  including  dementia, 
depression,  brain  injury,  and 
mental  retardation. 

The  QL  measures  were 
derived  from  data  embedded  in 
the  baseline  interview.  The 
measures  consist  of  responses 
by  participants  to  single  items 
or  questions,   (eg:  number  of 


reported  medical  conditions, 
frequency  of  getting  out  of  the 
house) ,  or  scales  we  developed 
from  responses  to  several 
questions,  (eg:  number  of 
friends,  contact  with  children 
and  organizational  particip- 
ation) ,  or  scales  composed  of 
several  items,  eliciting 
responses  to  related  questions 
(eg:  subjective  health, 
positive  affect,  intimacy  and 
social  support) .  Items  in  each 
of  these  scales  was  given  equal 
weight  in  arriving  at  the  scale 
score. 

In  addition  to  these  QL 
and  Cognitive  Impairment 
variables,  other  variables 
included  were  age  (in  5  year 
intervals),  income  (in  $1,000 
units) ,  gender  (male,  female) , 
education  (<than  or  >9+  years) , 
marital  status  (currently 
married  or  not) ,  living 
arrangements  (alone,  with 
others) ,  race  (white,  other 
than  white) ,  vision  defect 
(yes,  no) ,  hearing  defect  (yes, 
no)  . 

Analyses 

Two  main  statistical 
analytic  procedures  were 
performed.  First,  chi  square 
analyses  of  differences  between 
the  three  cognitive  impairment 
subgroups  (severe,  mild/ 
moderate  and  unimpaired)  in  the 
distribution  of  responses  to 
each  of  the  measures  in  the 


four  QL  domains.  The  results  of 
these  calculations  are  shown  in 
univariate  Table  1.  In  our 
earlier  work  we  noted  that 
sociodemographic  and  related 
differences  between  these 
impairment  groups  modified  the 
direction  of  apparent 
differences  in  mortality  rates 
between  impaired  groups5. 
Accordingly  we  developed  multi- 
variate models  to  assess  the 
association  between  the  QL 
measures  and  levels  of 
impairment  while  examing  the 
effects  of  those  variables  that 
differed  significantly  between 
the  three  impairment  groups. 
Ordinary  and  ordinal  multiple 
logistic  regression  analyses 
were  employed  for  this  purpose. 

Results 

The  results  of  the  Chi 
Square  analyses,  shown  in  Table 
1,  reveal  significant  differ- 
ences between  either  two  or 
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among  all  three  of  the 
impairment  subgroups  in  15  of 
the  17  QL  measures.  There  were 
no  significant  differences 
between  the  three  groups  in  the 
number  of  reported  medical 
conditions  and  behavioral 
problems. 

The  overall  pattern  of 
differences  reveal  that  among  6 
QL  measures  differences  were 
evident  among  the  3  subgroups 
of  impaired  persons  while  for  8 
other  measures,  it  was  the 
severely  impaired  group  which 
differed  significantly  from 
both  the  mild  and  the  un- 
impaired groups.  The  mildly  and 
unimpaired  groups  were  not 
significantly  different  from 
each  other.  There  were  only 
two  QL  measures  in  which  the 
severely  impaired  group 
differed  from  the  unimpaired 
group  but  not  from  the  mildly 
impaired  ("organizational 
participation"  and  "working") . 

There  were  some  differ- 
ences in  the  pattern  of 
impairment  group  differences 
among  the  other  QL  domains.  In 
the  QL  domain  "functioning", 
the  three  QL  measures  revealed 
significant  differences  among 
the  3  impairment  groups.  In  the 
QL  domain,  "Personal 
relationships,"  three  of  the 
four  measures  indicated 
differences  between  the 
severely  impaired  group  and  the 
other  2  groups.  Differences 
between  the  mildly  impaired  and 
unimpaired  groups  were  not 
significant.  The  patterns  of 
differences  in  the  "Health"  and 
"Activities  outside  the  Home" 
QL  domains  were  mixed.  The  two 
QL  measures  that  did  not  reveal 
significant  differences  among 
the  three  groups  were  measures 
located  with  the  "Health 
domain" . 

The  overall  pattern  of 
significant  QL  differences 
among  the  three  impairment 
groups,  evident  from  these 
results,  suggests  that  with 
some  few  exceptions,  it  is  the 
severely  impaired  who  bear  the 
burden  of  a  constricted  Quality 
of  Life,  most  markedly  in  the 
domains  of  functioning  and 
personal  relationships.  It  is 
also  evident  that  in  the 
domains  of  "health"  and 
"activities  outside  of  the 
home"  ,  and  in  the  domain  of 
"personal    relationship"    as 


well,  that  the  guality  of  life 
of  mildly  impaired  and 
unimpaired  persons  are  not 
markedly  different  from  each 
other  though  they  each  differ 
from  the  severely  impaired 
group. 

To  what  extent  are  these 
univariate  results  due  to  or 
dependent  upon  the  effects  of 
those  variables  upon  which  the 
three  impairment  groups 
differed?    A    mulitvariable 

analytic  approach  in  which 
potential  confounders  are 
controlled  is  reguired  to 
assess  the  independent  effect 
of  the  QL  measures.  We  applied 
multiple  logistic  regression 
procedures  for  the  purpose. 

The  results  cf  these 
analyses,  performed  for  each  of 
the  QL  domain  measures,  are 
summarized  in  Tables  2  and  3. 
In  these  two  tables  the 
adjusted  odds  ratios  and  the 
95%  confidence  intervals  of  the 
significant  cognitive  impair- 
ment category,  as  well  as  the 
additional  statistically 
significant  variables  that 
emerged  are  presented. 
Cognitive  Impairment  was 
significant  in  13  out  of  the  18 
QL  measures  analyzed.  In  each 
of  these  analyses  differing 
clusters  of  variables  other 
than  the  cognitive  impairment 
category  also  emerged  as 
significant. 

These  results  paralleled 
the  univariate  findings  with  a 
few  differences.  Controlling 
for  the  effects  of  the 
potential  confounders 
implicated  cognitive  impairment 
in  two  measures  in  the  health 
domain,  behavioral  problems  and 
number  of  reported  medical 
conditions.  For  the  measure, 
"sleep  difficulty"  variables 
other  than  impairment  emerged 
as  significant.  In  the  domain, 
personal  relationships, 
variables  other  than  cognitive 
impairment  were  significant  for 
the  measures  of  "contact  with 
children"  and  "contact  with 
others" . 

The  effect  of  these  and 
other  modifications  assessing 
the  differences  among  the  three 
impairment  subgroups,  though  of 
interest  and  of  some  importance 
with  respect  to  the  specific  QL 
measures  involved,  does  not 
abrogate  the  earlier  overall 
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results  of  the  univariate 
analyses.  These  multivariate 
results,  in  fact,  reinforce  and 
also  refine  certain  of  the 
earlier  results,  namely  that  it 
is  the  severely  cognitively 
impaired  group  whose  quality  of 
life  is  most  negatively 
impacted.  This  is  more  clearly 
evident  in  the  health  and 
functioning  domains  than  in  the 
domains  of  personal 
relationships  and  activities 
outside  of  the  home.  In  the 
latter  two  domains,  while  the 
results  are  somewhat  more  mixed 
the  QL  of  the  mildly  impaired 
does  not  appear  to  be 
significantly  different  in 
these  measures  than  the  QL  of 
the  unimpaired.  Differences 
between  the  mildly  and 
unimpaired  groups  which 
surfaced  in  the  univariate 
analysis  did  not  remain 
significant  in  the  multivariate 
analyses. 

Conclusion 

The  results  of  this 
exercise  in  studying  QL 
differences  among  elderly 
community  residents,  some  with 
and  others  without  signs  of 
evident  cognitive  impairment, 
indicate  a  greater  restrictive- 
ness  among  those  with  severe 
impairment  particularly  in 
measures  of  health  and 
functioning.  This  perhaps  is  to 
be  expected.  This  finding  is 
not  as  clearly  evident  in  those 
domains  involving  personal 
relationships  and  in  activities 
involving  interactions  with 
organizations  and  groups.  This 
finding  perhaps  is  not 
expected.  Also  not  expected  was 
the  finding  that  the  QL  of 
mildly  impaired  persons  is  not 
markedly,  or  at  least, 
significantly  different  from 
the  QL  of  unimpaired  persons. 
Appearances   of   differences 


between  these  two  groups  are 
due  more  to  sociodemographic 
and  other  differences  rather 
than  levels  of  cognitive 
functioning. 

Note  should  be  taken  also 
that  though  severe  cognitive 
impairment  apparently  imposes 
significant  restrictions  in  QL, 
its  effects  vary  across  the  QL 
measures.  The  importance  of 
impairment  in  relation  to  QL 
varies  with  the  character  of 


the  specific  QL  measure.  This 
finding  may  suggest  avenues  of 
intervention  that  can 
ameliorate  the  QL  impact  of 
severe  cognitive  impairment  in 
specific  areas  though  perhaps 
not  uniformly. 

Finally,  it  should  be 
noted  that  this  study  was  based 
on  specific  measures  of  QL.  The 
data  were  taken  from  interviews 
with  participants  in  a 
longitudinal  study  of  aging  and 
health.  The  study  was  not 
specifically  designed  to  be  a 
study  of  quality  of  life.  In 
fact,  the  data  antedated  more 
recent  interest  in  QL  and  its 
measurement.  We  do  believe, 
however,  that  we  have  generated 
a  reasonable  set  of  QL  measures 
that  approximate  the  QL  of  the 
population  studied.  We  achieved 
modest  proportions  of  explained 
variances  in  these  analyses, 
not  only  because  of  the 
specific  selection  of  variables 
and  their  metrics,  but  also 
because  of  the  sparse 
literature  in  this  area  of 
study,  a  problem  we  share  with 
more  recent  QL  investigators. 
We  believe,  however,  that  we 
have  made  a  significant, 
although  still  modest 
beginning,  and  we  await  further 
investigation  into  those  issues 
surrounding  the  QL  of 
cognitively  impaired  older 
people. 
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Levels  of  Cognitive  Impairment  Compared  Acros 

Quality  of  Life  Domains  and  Measures 


Coqnitive    Impairment    Levels 

Quality   of    Life   Domain 
Health 

Severe  vs 
Hild 

Severe   vs 
Not 

Impaired 

Mild   vs 
Not 
Impa  ired 

No.    Reported   Med.    Cond. 

NS 

NS 

NS 

Subjective   Health 

• 

* 

* 

Sleep   Difficulties 

* 

4 

NS 

Behavior   Problems 

NS 

NS 

NS 

Positive   Affect/Mood 

* 

* 

NS 

Depression 

* 

* 

NS 

Functioninq 

Social    Support 

* 

* 

* 

Problems   of   Daily   Living 

* 

* 

* 

Nursing   Home   Resident 

* 

* 

* 

Personal    Relationships 

Number   of   Friends 

* 

* 

* 

Contact   with   Children 

. 

* 

NS 

Contact   with  Others 

* 

* 

NS 

Intimacy 

* 

* 

NS 

Activities   outside   of 
the   home 

Gets   Out   of   the   Home 

* 

* 

* 

Religious   Attendance 

* 

* 

NS 

Organizational 
Participation 

NS 

* 

NS 

Walking 

NS 

* 

NS 

Exercise 

*  * 

• 

NS 

*  Chi  Square  is  significant  at  p<.05;  NS=Not  Signi 

Tabic  2 

Results  of  Logistic  Regression  Analyses 
Health  and  Functioning  Domains 


Mini    Mental    State    Exami 

Quality   of    Life    Domain 

Odds   Ratio 

95% 

Confidence 
Inteval 

Other 

Significant 

Variables 

A.    Health 

Number   Reported   Medical 
Condition 

Not 

Impa  ired 
(1-37) 

1.22-1.54 

Men, 

education, 
vision   and 
hearing   defect 

Subjective   Health 

Mild   or   Not 
Impa  ired 
(1.75) 

1.41-2. 17 

Higher    income 
no  vision   or 
hearing   defect 

Sleep  Oifficultiee 

NS 



Men,    white,    no 
vision  or 

hearing   defect 

Behavior   Problems 

Severe 
(2.13) 

1.53-2.96 

Vision  and 
hearing   defect 

Positive  Affect/Mood 

Mild   or   Not 

Impaired 

(2.36) 

1.00-2.94 

Lives   with   men 
others,    no 
hearing 
defect, 
younger   age? 

Depression 

Severe 

{2.39) 

1.81-3.17 

Lives   alone, 
white,    hearing 
defect 

B.    Functioninq 

Social    Support 

Severe 
(1.66) 

1.33-2.07 

Older,    lower 
income,    women 
not   married, 
vision   and 
hearing  defect 

Problems   with   Daily   Activies 

• 

Severe 
(2.78) 

2.20-3.53 

Older,    lower 
income,    women 
not   married, 
vision   and 
hearing   defect 

Nursing   Home   Resident 

Hild    (1.81) 
and 

Severe 
(4.19) 

1.36-2.39 
2.83-6.21 

Older   age 
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Table  3 


Results  of  Logistic  Regression  Analyses 
Personal  Relationships  and  Activities  Outc 
of  Nursing  Home  Domains 

i Mini  Mental  State  Exam  i 
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Quality  of  Life  Domain 
C.  Personal  Relationships 

Odds  Ratio 

95% 

Confidence 
Interval 

Other  Significant  j 
Variables 

Number  of  Friends 

Mild  or  Not 

Impaired 

(2.21) 

1.79-2.74 

Younger  age, 
higher  income 

Contact  with  Children 

NS 

Women,  married, 
lives  alone 

Contact  with  Others 

NS 

Younger  age, 
women,  education, 
married,  lives 
with  others 

Intimacy 

Mild  or  Not 

Impaired 

(1.85) 

1.43-2.40 

Higher  income, 
women,  not 
married 

D.  Activities  Outside 
of  Home 

Gets  Out  of  Home 

Mild  or  Not 

Impaired 

(2.80) 

2.22-3.55 

Lives  alone, 
married  women 
higher  income, 
younger  age,  no 
hearing  defect 

Religious  Attendance 

Mild  or  Not 

Impaired 

(1.79) 

1.41-2.27 

Older  age,  men, 
education,  no 
hearing  defect 

Organizational 
Participation 

NS 
(p=.067) 

Older,  higher 
income, 
education, 
married,  lives 
alone,  non-white,  | 
hearing  defect 

Working 

NS 

Younger  age, 
higher  income 

Exercise 

Mild  or  Not 

Impaired 

(2.94) 

2.15-4.02 

Younger  age, 
higher  income, 
men,  no  hearing 
defect 
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USE  OF  THE  1989  NATIONAL  LONG-TERM  CARE  SURVEY  FOR  EXAMINING 
COGNITIVE  IMPAIRMENT  ELIGIBILITY  CRITERIA 

Mary  E.  Jackson,  The  MEDSTAT  Group 
Pamela  Doty 


Analyses  of  the  1989  National  Long- 
Term  Care  Survey  (NLTCS)  and  its  compan- 
ion Informal  Caregiver  Survey  (ICS)  were 
used  to  examine  the  comparability  of  the 
cognitive  impairment  eligibility  criteria 
to  the  3+  ADL  criteria  in  the  Clinton 
Administration's  long-term  care  health 
reform  proposal  from  the  Health  Security 
Act.  Also  addressed  by  these  analyses  is 
the  extent  to  which  functional  indica- 
tors, i.e.  ADLs,  by  themselves,  are  suf- 
ficient for  identifying  both  the  physi- 
cally disabled  and  severely  cognitively 
impaired  population.  The  NLTCS  was  used 
because  it  provides  a  nationally  repre- 
sentative sample  of  Medicare  beneficia- 
ries age  65+  in  1989  that  allows  for  the 
generation  of  national  estimates  on  this 
population. 

The  Criteria.  The  Health  Security  Act 
(HSA)  specified  that  a  person  would  be 
eligible  for  long-term  care  benefits 
outlined  in  the  plan  if  s/he  was  disabled 
in  3  or  more  ADLs  out  of  five  (bathing, 
dressing,  toileting,  transferring,  eat- 
ing) or  experienced  a  comparable  level  of 
severe  cognitive  impairment.  The  bill 
stated  that  eligibility  based  on  severe 
cognitive  impairment  was  to  be  indicated 
by  a  standard  mental  status  protocol (s) 
indicating  severe  cognitive  impairment, 
PLUS  one  of  the  following:  1  or  more  ADLs 
out  of  the  core  five  (bathing,  dressing, 
toileting,  transferring,  eating) ;  a 
serious  behavior  problem (s)  that  creates 
the  need  for  supervision  to  prevent  harm 
to  the  individual  or  others;  or  an  IADL 
indicating  some  cognitive  impairment. 

Operational izing  the  Criteria.  First,  we 
attempted  to  operationalize  these  eligi- 
bility criteria  using  the  data  items 
available  in  the  1989  NLTCS  and  ICS.  The 
five  ADLs  specified  in  the  Health  Securi- 
ty Act  are  available  in  the  data  set; 
disability  in  each  ADL  was  defined  as 
chronic,  i.e.,  having  lasted  three  months 
or  longer  and  requiring  either  hands-on 
or  stand-by  assistance  for  performance. 

Cognitive  impairment  eligibility 
criteria  were  operationalized  using  the 
variant  of  the  Short  Portable  Mental 
Status  Questionnaire  (SPMSQ)  available  in 
the  1989  NLTCS.  This  battery  contains  10 
items,  9  of  which  are  identical  to  the 
SPMSQ.  The  one  item  that  differs  from  the 
SPMSQ  is  a  slightly  less  difficult  item 
than  appears  in  the  original  version  of 
the  instrument;  as  such  we  accept  a  score 
of  4  or  more  errors  (rather  than  5) ,  to 
indicate  moderate/severe  impairment  and  a 
score  of  6  or  more  to  indicate  severe 
impairment . 


We  were  also  able  to  operationalize 
the  additional  criteria  that  the  HSA 
stipulates  must  be  met  for  evidence  of 
cognitive  impairment.  As  previously  men- 
tioned, ADL  items  are  readily  available 
in  the  data  set  (for  operationalizing  the 
additional  criteria  of  1+  ADLs  in  the  CI 
criteria).  The  behavior  problem  items 
available  in  the  NLTCS  are:  wandering, 
frequent  temper  tantrums,  and  compulsive 
stealing.  Although  these  items  may  not 
be  those  one  would  choose,  they  were  the 
only  proxies  available  in  the  dataset, 
and  thus  were  used  to  operationalize  the 
behavior  problem  criteria.  And  finally, 
the  cognitive  IADLs  were  operationalized 
with  the  medication  management,  money 
management,  and  telephoning  items. 

Comparison  Standards.  In  order  to  assess 
whether  the  cognitive  impairment  eligi- 
bility criteria  is  comparable  to  the  3+ 
ADL  criteria  a  standard  of  comparison 
must  be  used  that  in  some  way  indicates 
the  level  of  disability  or  impairment, 
other  than  the  eligibility  criteria.  The 
NLTCS  provides  two  sources  of  standards . 
First,  is  the  number  of  hours  of  care  per 
week  an  individual  receives  from  all 
sources  —  from  informal  sources  (family 
and  friends)  who  are  unpaid  and  from 
persons  who  are  paid  to  provide  care. 
The  second  measure  chosen  to  be  used  as  a 
standard  is  an  item  in  ICS  which  asks  the 
caregiver  how  many  hours  the  disabled 
person  can  safely  be  left  alone. 

Results .  Table  1  shows  that  persons 
meeting  the  eligibility  criteria  of  3  or 
more  ADLs  receive  70  hours  of  care  per 
week,  on  average.  However,  this  estimate 
includes  persons  with  4  and  5  ADL  dis- 
abilities; because  such  individuals  have 
more  needs  and  therefore  receive  more 
hours  of  care,  the  average  number  of 
hours  is  somewhat  inflated.  In  order  to 
compare  to  the  CI  criteria  a  fairer  ap- 
proach is  to  use  the  number  of  hours 
associated  with  persons  having  3  ADLs, 
which  is  51.1  hours  per  week. 

Table  1  also  shows  that  regardless 
of  how  CI  is  operationalized,  more  loose- 
ly or  more  stringently,  the  number  of 
hours  that  those  with  cognitive  impair- 
ment receive  is  always  less  than  the 
number  of  hours  those  with  3  or  more  ADL 
disabilities  receive.  The  measure  that 
comes  the  closest  to  being  comparable  is 
the  one  that  combines  behavioral  evidence 
for  the  need  for  care/oversight  with  6  or 
more  errors  on  the  SPMSQ.  This  CI  mea- 
sure is  the  most  stringent  as  it  requires 
evidence  of  more  severity  on  the  mental 
status  protocol . 
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It  should  also  be  noted  that  the 
SPMSQ  as  a  stand-alone  criteria  yields 
the  poorest  comparison.  But  when  the 
protocol  is  combined  with  some  manifes- 
tation for  the  need  for  care,  comparabil- 
ity is  enhanced. 

Similar  results  are  obtained  when 
the  comparison  standard  is  the  number  of 
hours  an  individual  can  safely  be  left 
•alone.  Those  with  3  ADL  disabilities  can 
be  left  alone  for  6.1  hours,  on  average 
whereas  those  with  6  +  MSQ  errors  and  some 
evidence  of  the  need  for  care  can  be  left 
alone  for  6.7  hours,  on  average. 


Table  1 

Total  Hours/Week  of  Care 

and  Hours  Alone 

by  Various  Disability  Criteria 

Disability 
Criteria 

Mean  Hours 
of  Care 

Mean  Hours 
Alone 

3+  ADL 

70.0 

4.1 

1  ADL 

23.2 

12.0 

2  ADL 

33  .7 

7.7 

3  ADL 

51.1 

6.1 

4  ADL 

63.1 

4.7 

5  ADL 

88.4 

1.9 

4+  SPMSQ 
Errors 

35.2 

9.0 

6+  SPMSQ 
Errors 

41.7 

7.3 

4+  SPMSQ 
Errors 

AND 

One  of  the 
following: 

•1+  ADL 
•Behavior 

Problem (s) 
•Cognitive 

IADL ( s ) 

45.4 

8.4 

6+  SPMSQ 
Errors 

AND 

One  of  the 
following: 

•1+  ADL 
•Behavior 

Problem (s) 
•Cognitive 

IADL(s) 

50.1 

6.7 

Table  2  displays  the  population 
estimates  associated  with  the  3+  ADL 
disability  criteria  and  operationalized 
HSA  cognitive  impairment  criteria  in  con- 
junction with  the  3+  ADL  criteria.  The 
estimates  are  for  the  1989  aged  65  and 
older  population.    They  indicate  that 


close  to  one  million  elders  (or  3.2%  of 
the  65+  population)  would  be  eligible  for 
benefits  if  the  sole  criteria  were  3+  ADL 
disabilities.  When  the  cognitive  impair- 
ment criteria  are  added,  an  additional 
1.3  to  2.0  percent  of  the  population 
would  be  eligible,  depending  upon  the 
exact  definition  of  cognitive  impairment. 


Table  2 

Number  and  Percent  of  Aged  65+ 

Meeting  Various  Eligibility 

Criteria 

Criteria 

Population 
Estimate 

%  of  65+ 
Population 

3+  ADL 

999,263 

=  3.2 

3+  ADL 

OR 

4+  SPMSQ 

Errors 

AND 

One  of  the 
following : 

•1+  ADL 
•Behavior 
'  Problem(s) 
•  Cognitive 
IADL(s) 

1,638,262 

=  5.2 

3+  ADL 

OR 

6+  SPMSQ 

Errors 

AND 

One  of  the 
following: 

•1+  ADL 
•Behavior 

Problem (s) 
•Cognitive 

IADL(s) 

1,409,233 

=  4.5 

There  are  obviously  some  individuals 
who  will  meet  both  the  3+  ADL  and  CI 
eligibility  criteria.  Of  those  with  3  or 
more  ADL  disabilities,  between  one-third 
(33.1%)  and  two-fifths  (41.7%)  are  also 
cognitively  impaired,  depending  upon  how 
CI  is  operationalized.  And  of  those  who 
meet  CI  criteria,  approximately  2/5  are 
also  disabled  in  3  or  more  ADLs . 

In  order  to  examine  whether  ADLs 
might  suffice  as  a  stand-alone  criteria 
for  identifying  both  the  physically  dis- 
abled and  cognitively  impaired,  two  'mul- 
tiple regression  analyses  were  conducted, 
one  with  the  dependent  variable  being 
total  hours  of  care,  and  the  other  hours 
the  person  could  safely  be  left  alone. 
The  number  of  ADLs  and  the  combined  CI 
measures  were  entered  as  independent 
variables  along  with  several  control 
variables  that  are  known/suspected  to 
affect  how  much  care  a  person  receives. 
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The  control  variables  included  the  care 
recipient's  age,  sex  and  race;  the  pri- 
mary caregiver's  age,  sex  and  relation- 
ship to  the  care  recipient,  i.e.,  spouse 
or  not . 

Table  3  presents  the  regression 
coefficients  associated  with  the  two  key 
independent  variables,  i.e.,  ADLs  and 
cognitive  impairment.  (Coefficients  for 
the  control  variables  are  not  shown) . 
These  coefficients  can  be  interpreted  as 
the  number  of  hours  of  care  (or  hours  the 
care  recipient  can  be  left  alone)  associ- 
ated with  each  measure.  Results  indicate 
that  for  each  additional  ADL  disability 
there  is  an  increase  of  11.6  hours  of 
care.  Cognitive  impairment,  on  the  other 
hand,  is  associated  with  a  decrease  of 
3 . 2  hours . 


Table  3 . 
Predictors  of  Total  Hours  of  Help 
and  Hours  Person  Can  Be  Left  Alone 

Independent 
Variables 

Care 

Alone 

#  ADL/5 

* 
11.6 

* 
-3  .1 

Cognitive 
Impairment 

-3.2 

* 

-4.1 

Interaction 

3  .5 

0.8 

develop  reasonable  cognitive  impairment 
criteria  comparable  to  the  3+  ADL  crite- 
ria, particularly  if  the  cognitive  im- 
pairment criteria  are  based  upon  indica- 
tors that  reflect  true  need  for  care 
and/or  oversight  as  manifested  by  func- 
tional and/or  behavioral  factors.  Never- 
theless, the  results  also  suggest  that 
ADL  criteria  alone  are  not  sufficient  for 
identifying  the  severely  cognitively 
impaired,  even  when  disability  in  an  ADL 
is  defined  to  include  stand-by  assistance 
and  supervision  (in  addition  to  hands-on 
assistance) .  However,  other  function- 
based  measures  (e.g.,  some  combination 
of  ADLs  and  IADLs)  may  prove  effective  in 
identifying  this  population. 

Finally,  one  must  remember  that 
these  analyses  were  based  on  survey  data, 
and  that  several  of  the  items  used  to 
operationalize  the  eligibility  criteria 
are  proxies  for  indicators  that  would  be 
employed  in  determining  program  eligibil- 
ity. Moreover,  even  if  there  were  a  one- 
to-one  correspondence  between  the  survey 
items  and  eligibility  criteria  items,  one 
would  still  have  to  offer  the  caveat  that 
even  the  most  valid  criteria  can  identify 
an  unintended  population  —  in  this  in- 
stance the  less  than  severely  cognitively 
impaired  —  if  not  reliably  administered, 
or  administered  as  intended. 


p  <  .05 

An  interaction  term  was  also  includ- 
ed in  the  model.  It  reflects  the  number 
of  hours  a  person  who  is  both  cognitively 
impaired  and  ADL  disabled  receives.  The 
results  indicate  those  who  are  cogni- 
tively impaired  received  3.5  additional 
hours  of  care  for  each  additional  ADL 
disability. 

Note  that  the  ADL  coefficient  is  the 
only  one  that  is  statistically  signifi- 
cant. Cognitive  impairment,  once  ADLs 
are  taken  into  account,  does  not  add 
significantly  to  the  hours  of  care,  and 
neither  does  the  interaction  term.  ADLs 
are  clearly  the  strongest  predictors  of 
the  need  for  care. 

However,  the  results  are  somewhat 
different  when  the  outcome  is  the  hours 
alone  variable.  Interestingly,  each 
additional  ADL  is  associated  with  a  per- 
son being  able  to  be  left  alone  for  3.1 
fewer  hours,  but  cognitive  impairment  is 
associated  with  4.1  fewer  hours.  And 
coefficients  for  both  the  ADL  and  cogni- 
tive impairment  variables  are  statisti- 
cally significant,  suggesting  that  ADL 
criteria  alone  is  not  sufficient  for 
predicting  level  of  care  need. 

Discussion.   The  results  of  these  analy- 
ses indicate  that  it  may  be  possible  to 
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ANALYTICAL  METHODS  FOR  DISEASE  PATTERNS  AND  SOCIOGEOGRAPHY : 
ECOLOGICAL  APPROACHES 

Deborah  Wallace,  Ph.D.,  Consumers  Union 


Introduction 

Public  health  examines  populations  and 
their  health  status,  thus  qualifying  more  as 
an  ecological  discipline  than  a  purely 
medical  science.   Adaptation  of  ecological 
analytical  approaches  can  yield  valuable 
results  in  socicgeographic  public  health 
research. 

Tuberculosis:  A  Case  History  of  Methodology 
Transfer  from  Ecology 

How  to  analyze  geographic  differences 
in  population  health  patterns?   Figure  1 
displays  the  1990  tuberculosis  incidences  of 
the  30  health  districts  of  New  York  City. 
Note  two  bands  of  especially  high  incidence: 
Southwest  Bronx-Harlem  and  Lower  Eastside- 
Brooklyn  corner  near  Manhattan. 

John  Sbarbaro  (1991)  stated  that 
blacks  are  genetically  susceptible  to  TB. 
This  statement  lets  public  health  officials 
off  the  hook  for  high  TB  rates  in  segregated 
areas,  despite  the  fact  that  the  high  rates 
were  the  basis  of  Sbarbaro 's  statement. 
Figure  2,  the  major  racial  components  of  the 
59  NYC  community  districts  (CD's)  in  1990 
(NYC  Planning  Dept,  1994)  indicates  that 
racial  composition  cannot  explain  the  1990 
pattern. 

Historically,  the  Tuberculosis  Leagues 
explored  the  influence  of  household  crowding 
on  TB  occurrence  (Wichen,  1936) .  The  Index 
of  Extreme  Housing  Overcrowding  of  the 
Census  Bureau  (percent  of  dwelling  units 
with  1.51  persons  per  room  or  more)  of  the 
health  districts  fits  incidence  pattern 
imperfectly  bu   bettt-r  than  race  (figure  3)  : 
R2  of  nearly  50%.  Drucker  et  al  (1994)  found 
housing  overcrowding  was  an  important  factor 
in  household  TB  transmission. 

Income  level,  not  surprisingly, 
coincides  with  the  TB  pattern  (figure  4, 
1990  median  incomes  of  the  CD's,  from  New 
York  Planning  Dept.,  1994)  better  than  race. 
Rent  imperfectly  indicates  income.   In 
ecology,  we  use  indices  regularly.   The 
reciprocal  of  average  rent  indicates 
functional  average  poverty.   We  use  this 
index  because  we  have  the  average  rent  data 
for  the  30  health  districts  but  not  the 
poverty  levels  or  median  incomes.   Indices 
may  be  necessary  in  the  use  of  easily 
available  administrative  data  sets.   The  R2 
of  this  poverty  index  regressed  against  1990 
TB  incidence  is  over  40%,  close  to  that  of 
housing  overcrowding.   But  regression  of  the 
index  of  extreme  housing  overcrowding 
against  the  index  of  poverty  yields  an  R2  of 
only  19%,  a  hint  that  the  two  factors  are 
semi-independent . 

Although  loss  of  control  over  TB  began 
in  1975-1976,  continuous  epidemic  process 
began  in  1979.   In  fact,  1978  was  the  year 
of  lowest  historic  citywide  incidence. 
After  1978,  TB  spread  through  the  City 
except  for  the  wealthiest  neighborhoods 
(Figure  5  and  Table  1) .  Table  1  shows  that 
only  the  wealthiest  health  district  escaped 
the  epidemic.   The  corresponding  community 


district  with  a  median  income  of  $53K  also 
has  a  high  income  diversity,  with  over  15% 
of  the  households  earning  over  $150K  per 
year.   Diversity  measurement  was  performed 
on  the  CD's  with  the  Shannon-Weaver  Index  of 
Diversity  (Magurran,  1988)  .  In  ecology,  we 
developed  methods  of  measuring  diversity 
decades  ago. 

For  a  detailed  review  of  events 
leading  to  creation  of  essentially  citywide 
susceptibility  to  TB,  see  Bulletin  of  the 
NYAM,  66,  Sept/Oct.  1990  (Wallace  and 
Wallace,  1990) .   The  poor  neighborhoods  of 
color  were  targeted  ,  beginning  in  1969,  by 
successive  mayors  implementing  a  policy  of 
"planned  shrinkage"  which  took  away  housing 
preservation  services  such  as  fire  control . 
Fire  and  building  abandonment  flared  into 
literal  epidemics,  parasites  on  the  housing 
stock.   The  Census  Bureau  counted  the 
dwelling  unit  loss  between  1970  and  1980  in 
the  hundreds  of  thousands  (fig. 6:  each 
blackened  area  is  many  census  tracts  each  of 
which  lost  500  or  more  dwelling  units) . 
Mass  migration  resulted,  a  disruption  on  the 
scale  of  low  level  war  or  famine.   Citywide 
TB  followed  the  trends  of  housing  over- 
crowding (fig. 7)  during  the  game  of  "musical 
homes",  and  after  1980,  homelessness  became 
rampant  (Wallace,  1989  and  1990) . 

Area  population  density  influences 
community  transmission  because  of  its 
determination  of  active  case  density  and 
density  of  susceptibles .  Table  2  shows  the 
TB  cases  per  square  mile  of  selected  health 
districts  in  1978  and  1990.  Eighty  percent 
of  the  1990  incidence  pattern  can  be 
explained  in  a  stepwise  regression  with 
independent  variables  of  1978  case  density 
(initial  condition)  and  the  1990  index  of 
poverty  (1/average  rent) .   At  this  scale, 
two  types  of  transmission  obviously  occur: 
household  and  community. 

The  1970 's  saw  the  most  rapid  city-to- 
suburb  migration  ever,  with  1.3  million 
whites  leaving  NYC  for  the  suburbs. 
Suburbanites  kept  the  jobs  in  the  City,  and 
number  of  jobs  in  the  City  grew  through 
1990.  In  1980,  1,744,433  people  worked  in 
Manhattan;  by  1990,  the  number  had  risen  to 
2,039,651  (Census  Bureau,  Journey-to-Work 
data)  .   In  1980,  the  density  of  workers  per 
square  mile  in  Manhattan  was  61424;  in  1990, 
it  was  71819.   Manhattan  was  and  is  the 
heart  of  the  foraging  range  of  24  counties, 
about  20  million  people.  Figure  8  displays 
the  commuting  field  of  Manhattan:  percent  of 
a  county's  workforce  which  works  in 
Manhattan  vs  the  miles  between  Manhattan  and 
the  center  of  the  county.   Even  as  far  as  60 
miles  away  (Fairfield  County,  CT) ,  over  5% 
of  the  workers  had  jobs  in  Manhattan  in 
1990,  slightly  more  than  in  1980. 

Commuting  indicates  city-suburb 
contact.   Indeed,  the  regional  structure  of 
TB  incidence  1985-1992  reflects  the 
proportion  of  workers  commuting  into 
Manhattan  and  the  poverty  rates  of  the 
counties  (fig.  9).   For  details  of  the 
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composite  index  of  commuting  and  poverty, 
see  Wallace  and  Wallace  (1995) .  The  TB 
incidence  of  the  City,  determined  by  the 
planned  shrinkage  public  policy,  in  turn 
determines  the  suburban  counties'  TB 
incidences,  modulated  by  local  poverty 
level . 

Processes  beginning  at  the  scale  of 
the  neighborhood  reached  down  into  the 
households  and  up  into  the  whole  City  and 
the  entire  metro  region.   A  recent  court 
decision  (Rev.  Monsignor  John  J.  Powis  et  al 
vs  Rudolph  Giuliani  et  ano,  Index 
#100990/95,  Supreme  Court,  State  of  NY,  NY 
County)  brought  hard  municipal  service 
allocation  into  the  environmental  realm. 
Inequalities  in  such  services  must  be  seen 
as  issues  in  environmental  justice. 

Table  3  shows  that  NYC  is  not  a 
special  case.   Five  of  the  eight  analyzed 
metro  regions  show  regionalization  of  TB, 
i.e.  commuting  patterns  correlate  highly 
with  TB  incidence  patterns.  Table  3  also 
compares  regressions  of  poverty  and 
overcrowding  vs  TB  incidence  for  Detroit, 
Philadelphia,  San  Francisco,  Washington,  and 
NYC.   Overcrowding  affects  little  of  the  TB 
pattern  in  Philadelphia,  an  effect  which  may 
account  for  only  a  trend  to  regionalization. 
Detroit  has  the  lowest  number  of  jobs  per 
square  mile  in  the  central  county  and  may 
not  be  the  site  of  major  work-related 
transmission.  The  socioeconomic  and 
demographic  structure  of  each  metro  region 
determines  the  strength  of  regionalization. 
The  housing  overcrowding,  poverty,  and  job 
density  of  the  inner  city  determines  whether 
regionalization  of  TB  will  occur  at  all.  At 
the  regional  scale,  three  types  of 
transmission  must  occur:  household, 
community,  and  work-related,  loosely  meant. 

We  have  used  tools  from  the  ecological 
toolbox  to  look  at  pattern  of  TB  incidence 
and  its  possible  determinants:  measurement 
of  diversity,  mapping,  correlations  of 
phenomena  which  co-vary  within  the  same 
geographic  area,  and  changes  in  geographic 
patterns  over  time.   A  last  example  applies 
the  Niche  Exclusion  Model  to  TB  in  the  52 
CDC  designated  "states"  (the  states  plus  DC 
and  NYC) .  The  Niche  Exclusion  Model  finds 
the  species  in  an  ecological  community  which 
compete  for  the  same  resources . 

Figure  10a  displays  rank  vs  the 
detrended  TB  incidences  for  years  1985,  1987 
and  1990  for  the  top  25  states.   Figure  10b 
displays  the  same  data  for  AIDS.   Figure  10c 
shows  the  same  for  measles.  Detrending  was 
performed  by  divided  each  state's  incidence 
by  the  average  for  all  52  states  for  the 
given  year (incidences  from  CDC's  Annual 
Summaries  of  Reportable  Diseases  for  1985, 
1987,  and  1990) .  We  know  that  AIDS  is  a  new 
epidemic  spreading  from  epicenters.  The 
evening  out  of  the  rank  vs  incidence 
documents  this  spread,  as  more  states  get 
drawn  into  the  epidemic  process.  Measles 
shows  this  evening  out  more  strongly.   The 
rise  in  TB  incidence  showed  the  opposite 
behavior.  The  highest  ranking  incidence  rose 
farther  and  farther  above  the  lower  ranked 
ones.   Old  large  reservoirs  simply  grew  and 
left  the  others  behind. 

Figure  11a  graphs  rank  vs  the  log  of 
TB  incidence  for  the  same  years  as 


Figure  10.   Figures  lib  and  lie  show  the 
same  for  AIDS  and  measles.  In  1985,  the 
latter  two  diseases  had  a  pattern  of  little 
isolated  clumps  along  the  plot  line,  not  a 
single  system.   By  1990,  measles  had 
resolved  into  a  single  system  and  AIDS  had 
nearly  resolved  into  a  single  system,  as 
shown  by  a  nearly  smooth  curve.  TB,  on  the 
other  hand,  went  from  a  smooth  line  in  1985 
to  a  series  of  three  or  more  lines  with 
greatly  different  slopes.  What  had  been  a 
single  system  had  devolved  into  several 
subsystems.  The  states  along  these  straight 
lines  would  be  interpreted  by  the  Niche 
Exclusion  Model  as  "competing"  for  cases.  If 
cases  are  the  inverse  of  resources  needed  to 
prevent  TB,  the  interpretation  would  point 
to  competition  within  these  systems  for 
these  resources  (housing,  jobs,  working 
conditions,  access  to  health  care,  etc.). 
The  steepness  of  the  slope  indicates 
intensity  of  competition  for  scarce 
resources.  For  an  explanation  of  the  Niche 
Exclusion  Model,  see  Pielou  (1975)  or 
Magurran  (1988) . 

A  further  list  of  sources  for 
potentially  useful  approaches  and  analytical 
models  is  appended  to  this  paper.   Public 
health  scientists  should  be  encouraged  to 
transfer  methods  and  experiment.   We  would 
be  happy  to  serve  as  resources  for 
scientists  willing  to  try  out  these  methods 
but  wanting  some  help  getting  started. 
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Table   2 .    1990   New   Case    Incidence 
Health  District 


minus    1978    New   Case    Incidence   by 


health  district 

difference 

health  district   di 

f ference 

Central  Harlem 

181.2 

Bay  Ridge 

5 

East  Harlem 

110 

Bedford 

82.3 

Kips  Bay-Yorkville 

-4.9 

Brownsville 

34 

Lower  East  Side 

87.3 

Bushwick 

69.5 

Lower  West  Side 

26 

Flatbush 

25.3 

Riverside 

33.2 

Fort  Greene 

41.2 

Washington  Heights 

37.8 

Gravesend 

13.5 

Fordham-Riverdale 

20.8 

Red  Hook-Gowanus 

25.9 

Morrisania 

93.8 

Sunset  Park 

16.7 

MottHaven 

70.9 

Williamsburg- 

Pelhara  Bay 

21.5 

Greenepoint 

55.2 

Tremont 

63.3 

Astoria-L. I. C. 

36.7 

Westchester 

17.1 

Corona 

17.8 

Flushing 

10.3 

Jamaica  East 

31.3 

Jamaica  West 

15.8 

Maspeth-Forest  Hills 

5.1 

Richmond 

6.2 

Table  2.  TB  Cases  Per  Sq-Mile  by  Health  District 


Health  District 


1978 


1982 


1986 


1990 


Top  Five  Districts  in  1990  for  Incidence 


Central  Harlem 
Lower  Eastside 
East  Harlem 
Morrisania 
Bedford 


43 

66 

82 

144 

29 

34 

56 

89 

13 

18 

48 

81 

15 

18 

31 

45 

19 

21 

35 

55 

Bottom  Five  Districts  in  1990  tor  Incidence 


Gravesend 

Flushing 

Maspeth-Frst  Hills 

Richmond(SI) 

Bay  Ridge 

Kips  Bay-Yorkville 


2 

5 

5 

7 

1 

2 

1 

2 

2 

2 

2 

4 

<0.5 

1 

1 

1 

2 

3 

3 

4 

6 

5 

7 

4 
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Table  3.  Metropolitan  Regional ization  of  TB 

A.  Percent  R-sq  and  Significance  of  Regressions  without 
the  Travel  Center:  TB  Incidence  of  the  SMSA  Counties 
vs  Commuting  Index. 

The  travel  center  for  the  NY  metro  region  is  Manhattan. 


B 

oslon 

NYC 

R-sq 

52.9 

69.8 

signilicanc 

0.04 

0 

Philadelphia    Washington     Chicago 
24  70.8  60.2 


0.11 


0.001 


0.04 


Detroit 
10.1 


049 


St.  Louis 
25.3 

0.17 


San  Francisco 
58.2 

0  017 


B.  Possible  Factors  for  Strength  of  Regionalization 
in  Five  Metro  Regions:   housing  overcrowding, 
poverty  rates,  and  jobs  per  sq  mi  in  the  travel  center. 
The  travel  center  for  the  NYC  region  is  Manhattan. 

The  two  regressions  are  county  index  of  extreme  housing 
overcrowding  and  poverty  rate  vs  county  TB  incidence. 


NYC 


R-sq,  1985* 

69.4 

R-sq, 1985** 

68.1 

jobs/sq  mi,  1990 

71819 

Philadelphia  Washington  Detroit  San  Francisco 

16.9                   79.7  34.6                  30.8 

46.5                   444  27.5                  38.8 

5531                   11565  1372                  11904 


'Index  o(  Extreme  Housing  Overcrowding  vs  TB  incidence 
**  Poverty  Rate  vs  TB  incidence 


Geography  of  1990  TB  Incidence:  NYC  Health  Districts 


Community  Districts 
in  New  York  City 


Incidence  =  cases  per 


Dominant  Ethnicity 

A  =*  Asian 
B  =  black 
H  =  Hispanic 
W  =  white 


Figure  2.    Dominant  Ethnicities   of   the  59  Community  Districts   of 
NYC. 


Figure    1.    Ranges   of   Tuberculosis   New  Cases/100,000    In    1990   of   the 
30  Designated  Health  Districts   in  New  York  City. 
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Index  of  Extreme  Housing  Overcrowding:  Health  Districts 

Index  =  percent  of  dwelling  units  with  1.51  persons  or  more/room 


Community  Districts 
in  New  York  City 


symbol 


range  of  percent 


solid  black  > 

stripes  7-10 

"plus"  signs  6-7 

circles  5-4! 

boxes  4-5 

solid  white  1-4 


Figure  4.  1990  Median  Income  Ranges  of  Che  NYC  Community 
Districts. 


Figure  3.  1990  Ranges  of  Percent  of  the  Dwelling  Units  with  1.51 
Persons  per  Room  or  More  of  the  NYC  Health  Districts. 


Figure  5.  The  Spread  of  High  New  TB  Case  Incidence  in  NYC. 

Solid  black:  top  ranking  health  district 

Stripes:  other  members  of  top  quintile 

Stipples:  incidence  above  1978  citywide  incidence 

To  be  noted:  In  1979,  only  one  district  outside  Manhattan  was  in 

the  top  quintile;  by  1982,  the  top  quintile  was  dispersed  over 

three  boroughs.   By  1990,  Incidence  In  all  districts  of  the  Bronx 

and  all  districts  but  one  in  Brooklyn,  exceeded  the  1978  citywide 

incidence.   The  only  district  in  Manhattan  tc  r>..roin  at  low 

incidence  was  the  wealthiest  district  in  the  city.   Taken  from 

Wallace,  1994. 
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CHANGE  IN  HOUSING  UNITS:  1970-1SB0 
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Figure  6.  Massive  Housing  Unit  Loss  between  1970  end  1980.   Each 
black  area  is  composed  of  many  census  tracts,  each  of  which  lost 
over  500  housing  units  between  1970  and  1980  (Wallace  and 
Wallace,  1990) . 
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Figure  8.  The  Commuting  Field  of  Manhattan.   Proportion  of  the 
workers  living  in  the  NYC  SMSA  counties  who  work  in  Manhattan  vs. 
The  distance  (miles)  of  the  counter  center  from  Manhattan  center. 
Note  that  Fairfield  Cty.  (Over  60  miles  out)  has  more  than  5%  of 
its  resident  workers  commuting  to  Manhattan.   NYC  has  an 
effective  radius  of  >50  miles. 
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-13        -0.3        0.7         1.7         27 

0.183  Log(Mu/Area)  +  0.891  Log(Perpov) 

Figure  9.  Log  of  Incidence/100,000  of  TB,  AIDS,  and  Violent  Crime 

vs  Index  of  Commuting  (24  Counties  of  the  NYC  SMSA) . 

Top  graph:  Incidence  vs  only  Commuting  Index 

Bottom  graph:  Incidence  vs  Commuting  Index  and  Percent  of 

Population  in  Poverty 

For  commuting  index,  see  Wallace  and  Wallace,  1995. 
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Figure  10.  Detrended  1985,  1987,  and  1990  TB,  AIDS,  and  Measles 
Incidence  vs  Rank  for  Top  Ranked  25  CDC  "States".   Detrended 
incidence  =  state  incidence/national  mean.   Note  that  for  AIDS 
and  measles,  the  top  ranked  states  in  1990  fell  below  the  1985 
level,  especially  for  measles.   Also  the  "tail"  rose  by  1990. 
The  opposite  occurred  for  TB. 


Log  Measles  Cases  vs  Rank 


Figure  11.  Application  of  the  Niche  Exclusion  Model  to  Measles, 
AIDS,  and  TB:  Log  Incidence  vs  Rank  of  Top  Ranked  25  CDC  "States" 
for  1985,  1987,  and  1990.   The  marks  perpendicular  to  the  1985 
AIDS  and  measles  lines  bound  separate  little  systems  in  the  upper 
ranks,  especially  obvious  for  AIDS.   By  1990,  measles  had 
resolved  into  a  single  national  system,  and  AIDS  was  nearly 
similarly  resolved.   In  1985,  TB  was  more  a  single  system  than  in 
1990.   The  heavy  lines  in  the  TB  graph  connect  points  lying  along 
one  line  which  form  a  system:  ranks  1-4,  5-9,  and  10-14. 
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Log  AIDS  Cases  vs  Rank 
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THE  ROLE  OF  ETHNOGRAPHY  IN  PUBLIC  HEALTH 
by  Anne  Dievler 


I.  Introduction 

I  would  like  to  begin  my  presentation  with 
an  interchange  between  an  anthropologist  and  an 
epidemiologist : 

An  anthropologist  starts  by  saying:  "Many 
people  are  unhappy  with  the  U.S.  health 
care  system  and  are..."   And  an 
epidemiologist  interrupts  and  says:  "What 
people?  How  many?  When  do  they  feel  this 
way?   How  long  have  they  felt  this  way? 
Where  do  they  get  their  care?  What  kind 
of  care  do  they  get?   What  do  they  need?" 
So  the  anthropologist,  shrugs  her 
shoulders  in  frustration  and  says:  "Oh  — 
Never  mind. " 

While  there  may  be  some  irreconcilable 
covrununication  problems  between  anthropologists 
aril  epidemiologists,  today  I  hope  to  bridge  some 
of  that  gap  as  I  talk  with  you  about  one  of  the 
methods  that  anthropologists  use,  the 
ethnographic  method.   I  am  going  to  talk  about 
the  role  of  ethnography  in  public  health  and  its 
particular  usefulness  in  understanding  urban 
public  health  problems. 

II.  Defining  Ethnography 

First,  I  think  it  is  important  to  have  a 
clear  understanding  of  what  ethnography  is;  so 
if  those  of  you  that  are  medical  anthropolo- 
gists will  bear  with  me. . .  Ethnographic  research 
is  a  cyclical  and  iterative  process  (Figure  1) 
(Spradley,  1980;  Agar,  1980).   One  selects  a 
project,  develops  broad  research  questions  using 
a  conceptual  framework,  and  then  selects  a  site. 
Then  the  researcher  begins  to  ask  ethnographic 
questions  —  questions  not  unlike  those  asked  by 
our  epidemiologist  friend  here  —  but  questions 
that  are  perhaps  more  sensitive  to  cognitive 
processes  and  the  culture  of  the  individuals. 
In  addition  to  questions  of  what,  how,  where, 
the  ethnographer  asks,  like  a  two  year  old: 
why,  why,  why? 

The  ethnographer  then  collects  a  wide  range 
of  ethnographic  data  through  observing, 
interviewing,  reviewing  documents,  and  makes  an 
ethnographic  record   —  or  "f ieldnotes. "   Next 
the  researcher  analyzes  the  fieldnotes,  begins 
to  write  the  ethnographic  story  and  then 
reformulates  the  questions  with  insights  gained 
from  the  field  and  new  perspectives  from  the 
theoretical  literature,  and  then  goes  back  to 
the  field  once  again. 

Overall,  the  ethnographic  method,  while 
very  intensive  and  time-consuming,  can  provide 
rich  detail  —  and  what  has  been  called  "thick 
description"  (Geertz,  1973). 

I  would  like  to  emphasize  two  things  about 
this  research  cycle.   One  is  that  the  conceptual 
framework  is  very  important  —  for  example,  in 
studying  urban  problems  —  it  would  be  good  to 
have  some  theories  about  urbanization  —  whether 
it  is  the  deindustrialization  of  cities  or  the 
urban  underclass  —  or  if  one  is  looking  at 
urban  policy  making,  it  would  be  important  to 
look  at  what  the  literature  says  about  urban 
policy  making.   In  my  experience  in  public 
health,  we  are  often  weak  on  the  theory  side  of 
things. 


I  would  also  like  to  emphasize  that  the 
ethnographic  write-up  is  very  important.   The 
write-up,  also  called  an  "ethnography,"  is 
shaped  by  the  author's  theoretical  orientation 
and  the  narrative  and  rhetorical  conventions  he 
or  she  chooses  to  employ. 

III.  What  is  the  role  of  ethnography  in  public 
health? 

The  best  way  to  approach  this  question  is 
to  look  at  the  core  public  health  functions 
defined  by  the  Institute  of  Medicine, 
assessment,  assurance,  and  policy  development, 
and  to  see  how  ethnography  applies  to  each  area. 

A.  Assessment 

With  the  assessment  function  of  public 
health,  we  try  to  understand  health  problems  and 
assess  the  needs  and  health  status  of  the 
population. 

Many  of  the  most  pressing  urban  public 
health  problems:  violence,  teen  pregnancy, 
substance  abuse,  TB,  and  HIV/AIDS  are  complex 
problems  that  are  intertwined  with  social  and 
economic  conditions  such  as  poverty, 
homelessness,  and  unemployment. 

Because  ethnography  takes  a  holistic  and 
contextual  approach  to  problems,  it  can  help  us 
to  understand  these  problems  better.   Armed  with 
epidemiology  and  biostatistics,  public  health 
professionals  usually  select  a  sample,  develop  a 
questionnaire,  administer  and  then  analyze  it. 
Ethnography  can  help  us  to  figure  out  if  we  are 
asking  the  right  questions  and  if  we  are  asking 
them  in  the  right  way. 

Ethnography  has  also  been  practical  in 
helping  to  enumerate  populations.   For  example, 
you  are  all  familiar  with  the  fact  that  the  1990 
census  was  criticized  for  resulting  in  an 
undercount  of  the  population,  particularly  in 
urban,  inner  city  areas.   So  the  Census  Bureau 
hired  anthropologists  who  went  out  and  estimated 
the  extent  of  the  undercount,  and  the  causes  of 
the  undercount,  and  did  a  reenumeration.   As  a 
result,  fundamental  changes  were  made  in  the  way 
the  Census  Bureau  will  carry  out  the  year  2000 
population  survey. 

B.  Assurance 

The  assurance  function  of  public  health  has 
to  do  with  assuring  that  people  have  access  to 
care  and  that  they  get  quality  care. 

In  the  field  of  medical  anthropology,  there 
has  been  extensive  research  on  the  individual, 
doctor-patient  relationship  and  the  quality  of 
care  that  patients  receive  in  health  care 
settings.   I  think  there  has  been  less 
application  of  ethnographic  methods  to  the 
system  of  health  care. 

Recently  I  came  across  a  wonderful 
ethnographic  account  of  the  failure  of  urban 
health  care  entitled:  Mama  Might   be  Better  Off 
Dead   —  by  Laurie  Kay  Abraham  (1993).   Abraham 
is  actually  a  journalist,  but  like  an 
ethnographer,  she  spent  three  intensive  years 
chronicling  the  lives  of  a  sick,  poor  family  in 
Chicago  and  the  institutions  that  served  or 
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failed  to  serve  them.   As  a  result,  she  provides 
a  stunning  indictment  of  the  U.S.  health  care 
system.   I  think  public  health  would  benefit 
from  more  analyses  of  this  kind. 

C.  Policy  Development 

Finally,  how  useful  ia  ethnography  in 
studying  the  development  and  implementation  of 
public  health  policy  by  local,  state,  and 
federal  public  health  agencies  and  the  community 
agencies  they  work  with? 

Ethnographic  methods  have  been  used  since 
the  1930s  and  1940s  in  the  U.S.  to  understand 
the  culture  and  operation  of  organizations,  from 
the  classic  Hawthorne  studies,  to  studies  of 
public  sector  bureaucracies  such  as  Philip 
Selznik's  study  of  the  Tennessee  Valley 
Authority. 

In  my  own  work,  I  used  ethnography  to  study 
the  development  and  implementation  of  public 
health  policy  in  an  urban  bureaucracy,  the 
Washington,  D.C.  Commission  of  Public  Health. 

I  started  out  with  some  broad  research 
questions  —  I  wanted  to  know: 

Why  is  public  health  in  some  areas, 
particularly  urban  areas,  in  disarray? 

How  are  policies  being  formulated  and 
implemented  by  the  government  to  address 
major  urban  public  health  problems?   and 

What  is  the  role  of  rational  planning  in 
policy  making? 

From  these  broad  questions,  I  selected  the 
D.C.  Commission  of  Public  Health  as  the  site  for 
my  study,  and  I  also  selected  three  cases:  HIV 
prevention,  tuberculosis  control  and  childhood 
immunization. 

I  then  began  asking  ethnographic  questions 
and  doing  my  fieldwork.   The  whole  process 
continued  for  almost  two  years.   As  I  engaged  in 
this  ethnographic  research,  I  relied  heavily  on 
literature  from  the  fields  of  public  adminis- 
tration and  political  science;  because  early  on, 
I  recognized  that  rational  planning  was  not 
driving  the  policy  making  process,  but  rather 
what  was  important  was  the  nature  of  the 
bureaucracy  and  the  political  process. 

I  found  that  the  problems  that  I  studied: 
HIV/AIDS,  TB,  and  childhood  diseases  are 
persistent,  complex,  and  difficult  for  policy 
makers  to  solve.   Instead  of  being  a  rational 
process,  policy  making  was  slow,  incremental, 
reactive,  and  even  erratic.   Technical, 
bureaucratic,  and  political  factors  shaped  how 
problems  were  defined,  whether  they  got  on  the 
government's  agenda  for  action,  what  strategies 
were  developed  to  respond  to  the  problem,  and 
whether  or  not  those  strategies  were 
implemented. 

For  me,  using  ethnographic  tools  was  a  way 
to  try  to  make  sense  out  of  a  very  complicated 
policy  process.   Hopefully,  my  ethnographic 
write  up  —  which  I  did  in  the  form  of  case 
studies  —  will  also  be  useful  in  promoting 
understanding  about  why  public  health  is  in 


disarray  and  what  can  be  done  to  strengthen  the 
capacity  of  our  public  health  institutions. 


IV.  Conclusion: 

I  would  like  to  conclude  by 
attention  to  a  recent  article  by 
in  Social  Science  and  Medicine  ( 
points  out  that  there  are,  in  fa 
of  convergence  between  the  field 
pology  and  epidemiology  in  terms 
health  and  disease,  observationa 
and  the  broad,  interdisciplinary 
inquiry.  Both  fields  are  rooted 
tradition  of  understanding  how  t 
individuals  is  directly  affected 
physical,  social,  cultural,  and 
environments. 
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The  pressing  problems  of  urban  areas 
require  that  we  be  as  creative  as  possible  with 
our  methods,  and  I  think  that  ethnography  can 
compliment  many  of  our  traditional  epidemiologic 
approaches  and  provide  new  insights  in  public 
health. 
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METROPOLITAN  AREA  DATA  BASES  FOR  PUBLIC  HEALTH  ASSESSMENT 
Dennis  P.  Andrulis,  National  Public  Health  and  Hospital  Institute 

Paper  not  available  for  publication. 


203 


IDENTIFICATION 

OF 

DATA  NEEDS  TO  ADDRESS  THE  URBAN 

HEALTH 

CRISIS: 

A  POLICY  PERSPECTIVE 

Robert 

E. 

Full 

ilove, 

Columbia  University  School  of 

Public 

Health 

Pap 

er 

not  available  for  publication 

204 


Session  L 


measuring  neighborhood 

Effects  on  Health 


I 


~  .„_ »        \        '<■ 


p"*™  ***^«^*«w**v» . 


-t 


i...O;F 


■ 


EXAMINING  ALTERNATIVE  MEASURES  OF  UNDERSERVICE 


Thomas  C.  Ricketts,  Ph.D. 

University  of  North  Carolina  at  Chapel  Hill 

Donald  H.  Taylor,  Ph.D. 


The  federal  government  and  most  states  use  some 
form  of  indicators  to  identify  areas, 
populations,  or  institutions  as  medically 
underserved  in  order  to  prioritize  the 
allocation  of  programs  or  resources.   The 
current,  most  commonly  used  measures  have  been 
criticized  as  not  being  precise  enough  in 
identifying  underservice  or  as  being 
inappropriate  for  the  programs  which  use  them. 
This  study  used  a  theoretical  model  to  construct 
alternative  measures  of  underservice  applicable 
to  whole  U.S.  counties.   One  of  the  new, 
alternative  measures  identified  657  whole 
counties  as  underserved,  this  approximates  the 
number  of  Health  Professional  Shortage  Area 
(HPSA)  designations  (717)  for  1990.   This 
alternative  measure  was  also  compared  with  the 
HPSA  and  the  Index  of  Medical  Underservice  (IMU) 
on  its  ability  to  identify  areas  in  which 
respondents  to  the  National  Health  Interview 
Survey  were  more  likely  to  report  access 
problems  or  lower  health  status.   The 
alternative  measure  performed  as  well  or  better 
than  the  HPSA  or  IMU  in  predicting  individual 
problems  with  access  and  need. 

In  order  to  target  assistance  to  reduce 
medical  underservice  (defined  as  systematic 
access  barriers  to  primary  care) ,  the  federal 
government  has  relied  upon  the  Health 
Professional  Shortage  Area  (HPSA) ,  and  the  Index 
of  Medical  Underservice  (IMU)  methodologies  to 
identify  areas  eligible  to  receive  assistance 
under  various  programs  meant  to  improve  access 
to  health  care.   However,  the  continued  use  of 
both  the  HPSA  and  MUA  methodologies  has  been 
criticized  over  the  past  15  years  on  four  main 
fronts : 

(1)  areas  designated  by  these  methods  do 
not  differ  from  areas  not  so  designated 
on  key  outcomes  such  as  physician 
visits  and  health  status; 

(2)  they  lack  sufficient  theoretical 
justification  and  basis; 

(3)  they  do  not  clearly  define  what  it  is 
they  seek  to  measure;  and 

(4)  they  are  used  to  designate  areas  as 
eligible  for  programs  not  related  to 
the  original  intent  of  the  measures. 

These  critiques  are  causing  policy  makers  to 
explore  the  potential  for  the  combination  of  the 
HPSA  and  IMU  into  an  improved  measure  or  to 
create  new  measures  that  are  applicable  to 
specific  program  needs,  in  developing  new 
identification  of  problem  areas.   The  Health 
Resources  and  Services  Administration  has  been 
studying  both  the  HPSA  and  IMU  methodologies  for 
the  past  2  and  1/2  years  with  a  view  toward 
developing  a  single  methodology  for  use  with  all 
federal  assistance  programs  aimed  at  improving 
access  in  local  areas. 

In  the  project  describe  here,  a  research 
team  at  the  University  of  North  Carolina  has 
undertaken  a  comprehensive  review  of  the  use  of 
the  HPSA  and  MUA  methodologies  and  developed 
several  alternative  measures  of  underservice. 
The  first  year  of  our  review  of  the  HPSA  and  IMU 
methodologies  produced  the  following 
conclusions : 


1 .  There  are  some  conceptual  problems  with 
both  the  HPSA  and  MUA  methodologies,  but 
changes  made  to  them  could  have  large 
consequences . 

2.  Definitions  of  'underservice'  and 
'provider  shortage'  should  be  made  clear 
when  developing  any  new  methodology. 

3.  Policy  goals  of  programs  should  fit  the 
definition  of  underservice/provider 
shortage  adopted  to  designate  areas 
eligible  for  programmatic  assistance. 

4.  The  unit  of  analysis  for  these  types  of 
methodologies  may  begin  to  shift  away  from 
counties  or  other  localized  geographic 
areas  and  toward  population  access  to  a 
larger  health  care  delivery  system  that 
may  be  geographically  dispersed. 

5.  The  biggest  practical  issue  for  any 
methodology  is  what  data  are  needed  and 
who  is  responsible  for  designation  of 
areas  as  eligible  for  assistance  using  the 
methodology . 

6.  Any  change  in  methodology  for  designating 
eligible  recipients  for  assistance  will 
probably  create  winners  and  losers,  making 
any  change  politically  difficult. 


These  6  conclusions  formed  the  starting 
point  of  our  efforts  to  develop  an  alternative 
means  of  designating  counties  as  medically 
underserved.   Our  efforts  to  develop  an 
alternative  means  of  designation  were  guided  by 
clearly  defining  the  term,  suggesting  a 
conceptual  model  of  medical  underservice,  and 
testing  several  representations  of  this  model 
with  LISREL®  using  1990  county  level  data  from 
the  Area  Resource  File.   Respondents  to  the  1990 
National  Health  Interview  Survey  (NHIS)  living 
in  counties  designated  as  medically  underserved 
by  each  of  the  designation  methods  are  compared 
to  those  respondents  living  in  counties  not  so 
designated  in  order  to  determine  if  they  differ 
on  several  measures  of  access  and  need.   This 
research  produced  five  alternative  measures  of 
medical  underservice  (The  Taylor  Indices  of 
Medical  Underservice)  that  have  been  compared  to 
each  other  as  well  as  the  HPSA  methodology  and 
the  IMU. 

Policy  Review 

Since  their  introductions  in  1974  (IMU) 
and  1978  (HPSA) ,  the  two  indicators  have 
undergone  some  minor  changes  in  their  structure 
but  they  remain  basically  the  same  as  when  they 
were  first  applied.   The  HPSA  measure  has  been 
changed  formally  by  legislation  and  regulation 
to  include  a  prioritization  process  while  the 
MUA  has  remained  essentially  a  ranking  system 
for  counties  with  a  liberal  cut-off  level  for 
qualification  as  a  medically  underserved  area. 
The  HPSA  depends  largely  on  the  physician-to- 
population  ratio  but  the  threshold  level  of  one 


207 


primary  care  physician  to  3,500  people  in  a 
"rational  medical  service  area"  can  be  modified 
down  given  high  levels  of  need  indicated  by 
infant  mortality  or  other  factors.   The  HPSA 
designation  process  has  been  largely  a  locally- 
initiated  process  with  the  federal  Division  of 
Shortage  Designation  responding  to  applications 
and  requiring  areas  to  re-apply  after  three 
years  of  designation.   The  Division  was  reported 
to  have  significant  lag  times  in  the  application 
and  appeals  process  during  the  recent  past  but 
has  cut  turn-around  time  for  decision  to  30  days 
in  most  instances.   The  designation  process 
created  a  need  for  a  response  on  the  part  of 
state  and  local  level  officials  who  sought 
designation  to  support  needed  primary  care 
services  in  selected  communities.  Officials  in 
certain  states  and  localities  became  adept  at 
identifying  and  supporting  applications  with 
timely  and  appropriate  data  while  other 
localities  were  unable  to  cope  as  effectively 
with  the  process.   This  created  a  situation 
where  certain  areas  of  the  nation  were  more 
likely  to  have  designations  than  others 
independent  of  the  local  situation. 

The  number  of  areas  designated  as  MUAs 
remained  fairly  stable  over  the  period  1974  to 
1995;  there  has  not  been  a  comprehensive 
designation  of  counties  since  1981  and  fairly 
few  changes  in  the  listing  of  MUAs  are  made 
every  year  through  application  to  the  Division. 
There  has  been  substantial  changes  in  the  number 
of  HPSA  areas,  populations,  and  facilities 
designated  with  a  steady  upward  trend  in  the 
total  appearing  since  1990.   A  significant 
number  of  designations  have  been  continuous  or 
"chronic."   Fifteen  percent  of  352 
Nonmetropolitan  and  four  percent  or  3  0 
metropolitan  counties  have  been  designated  every 
year  since  1978.   Another  458  partial 
Nonmetropolitan  and  259  metropolitan  counties 
have  been  designated  more  or  less  permanently. 
Seven  hundred  forty-seven  counties  or  24  percent 
of  all  counties  have  never  been  designated. 


Table  1 


Chronic 

3  53 

3  0 

Whole 

(15%) 

(4.1%) 

Chronic 

458 

259 

Partial 

(19.5%) 

(33%) 

Non- 

997 

250 

Chronic 

(42.5%) 

(33.8%) 

Never 

535 

212 

Designated 

(22.8%) 

(28.7%) 

Total 

2343 

751 

(100%) 

(100%) 

The  MUA  is  a  score  assigned  to  areas  based 
on  four  factors:  Population-to-primary  care 
physician  ratio,  population  over  65,  infant 
mortality  rate,  and  percent  of  population  below 
poverty.   The  scoring  system  assigns  an  inverse 
weight  to  each  item  for  a  county  or  a  designated 
area  and  those  areas  which  have  a  score  of  62  or 
less  are  designated.   The  MUA  designation  is  not 
parallel  to  the  HPSA,  in  1980  123  Nonmetro  and  6 
Metro  counties  were  below  62  but  not  HPSAs.   In 
1990  these  numbers  dropped  to  82  Nonmetro  and  no 
Metro  counties. 


for  the  development  of  a  theoretical  model  of 
underservice.   The  literature  pointed  to  three 
general  factors  that  could  be  measured  at  the 
community  level  as  describing  underservice: 
Infrastructure,  Vulnerability,  and  Demand. 
Community  level  variables  were  used  to  confirm 
the  existence  of  these  factors.   Examples  of  the 
variables  used  include,  for  Vulnerability- 
percent  of  population  over  65,  mortality  rates, 
percent  below  poverty,  per  capita  income;  for 
Infrastructure— primary  care  providers,  medical 
specialists;  and  for  Demand— percent  population 
college  educated,  percent  with  white  collar 
jobs.   For  the  purposes  of  this  project,  all  of 
the  variables  related  to  the  individual  county 
since  these  data  were  the  most  readily  available 
and  all  variables  were  drawn  for  the  year  1990. 
The  project  recognized  that  rational  service 
areas  were  often  parts  of  counties  or  multiple 
counties  and  that  the  data  analysis  process  was 
one  of  estimation  and  approximation. 

A  model  containing  the  three  factors 
believed  important  in  defining  underservice  was 
estimated  using  the  LISREL  (structured 
equations)  technique  to  obtain  quantitative 
estimates  of  the  degree  of  "fit"  between  the 
health  system  (Infrastructure)  and  the  local 
population  (Vulnerability  and  Demand) .   The 
degree  of  "fit"  between  these  is  the  indicator 
of  medical  underservice  used  to  identify  places 
where  access  may  not  be  adequate. 

Figure  1:  Conceptual  Model  for  Determination  of 
Medical  Underservice 


The  model  depicted  above  produced  a  county  level 
score  for  "fit"  that  was  then  scaled,  this 
results  in  a  relative  measures  of  "service"  or 
access— counties  that  fell  into  the  bottom 
quartile  for  Nonmetropolitan  and  the  bottom 
decile  for  metropolitan  counties  on  the  Fit 
score  were  classified  as  "underserved. "  These 
cut  points  were  chosen  because  they  approximated 
the  cut-points  used  in  scales  to  determine 
underservice  in  the  HPSA  and  MUA  systems.   These 
criteria  identified  73  metro  and  584  Nonmetro 
counties  as  underserved  for  a  total  of  657 
counties.  This  compares  to  a  total  717  HPSA  and 
757  MUA  whole  counties  designated  in  1990.   The 
overlap  is  illustrated  in  Table  2. 


■QBH 

FIT 

IMU 

HPSA 

HPSA 

Model 

1990 

1990 

1992 

FIT  Model 

657 

IMU  1990 

430 

757 

HPSA  1990 

379 

422 

717 

HPSA  1992 

380 

421 

588 

722 

Construction  of   a  Theory  Baaed  Measure  of 
Undereervi ce 

The  second  year  of  the  project  began  with 
a  review  of  general  literature  on  need  and 
utilization  of  primary  health  care  as  the  basis 


208 


The  designation  pattern  based  on  FIT  identified 
several  geographic  areas  where  communities  may 
not  have  been  aggressive  in  obtaining  HPSA 
designation.   Areas  of  central  Kentucky,  west 
Texas,  northern  Missouri  and  southern  Iowa  were 


identified  by  the  FIT  method  as  being 
potentially  severally  undeserved  but  not 
designated  as  HPSAs. 

Comparison   of  FIT,    HPSA   and  MUA  designations   and 
Population  Based  Measures  of  Onderservlce 

There  are  no  locally  applicable  and 
consistent  measures  of  population  based  access 
or  underservice  with  which  to  verify  the 
accuracy  of  the  FIT  designation,  or,  for  the 
HPSA  and  MUA  process.   National  health  status 
and  perception  surveys  are  based  on  cluster 
sampling  and  are  meant  to  be  used  to 
characterize  the  nation  or  large  regions, 
occasionally  statistics  can  be  drawn  for  large 
states  or  for  cities.   In  order  to  compare  the 
performance  of  the  various  measures  of 
underservice  with  these  national  surveys, 
geographic  identifiers  have  to  be  attached  to 
the  individual  respondents  and  the  data 
aggregated  into  groups  for  those  respondents 
living  in  areas  designated  by  one  or  more  of  the 
measures.   This  was  done  using  the  1990  National 
Health  Interview  Survey  for  which  the  National 
Center  for  Health  Statistics  allowed  access  to 
the  individual,  geo-coded  records  under  a 
special  agreement. 

Within  the  question  structure  of  the 
National  Health  Interview  Survey,  there  is  no 
single  indicator  nor  is  there  an  apparent 
cluster  of  indicators  that  would  serve  as  a 
proxy  or  measure  for  access  or  underservice. 
The  analysis  then  was  to  determine  the  relative 
levels  of  correlation  between  the  three  methods 
(FIT,  MUA,  and  HPSA)  and  various  indicators 
thought  to  be  related  to  underservice.   These 
included  restrictions  on  activity,  presence  of 
chronic  conditions,  self -report  of  poor  health, 
and  use  of  physician  services.   The  analysis 
also  included  measures  of  intervening  variables 
generally  thought  to  affect  access  and  which  are 
directly  related  to  utilization  and  other 
indicators  of  outcomes  which  were  not  measured 
in  the  survey;  these  included  race,  income,  and 
education. 

The  analysis  revealed  that  the  persons 
living  in  HPSA,  MUA  and  FIT  designated  areas  had 
approximately  similar  characteristics  and  those 
characteristics  identified  them,  generally,  as 
being  more  underserved  than  people  not  living  in 
the  designated  areas.   However,  the  responses  of 
those  living  in  areas  designated  as  underserved 
did  not  differ  significantly  from  those  living 
in  areas  not  designated.   In  particular, 
consistent  differences  in  the  use  of  physician 
services  were  not  found.   This  agrees  with 
earlier  research  that  has  generally  concluded 
that  the  HPSA  and  IMU  do  not  do  a  particularly 
good  job  of  differentiating  areas  with  access 
problems  from  those  without.  These  measures  do 
appear  to  identify  people  with  health  care  needs 
and  different  health  status  but  they  do  not 
correlate  well  with  the  indicators  of 
utilization  which  is  often  considered  a  reliable 
indicator  of  access. 

Conclusions 

1.  There  are  objective,  theory  driven  ways  to 
identify  variations  in  areas. 

2.  Alternative  Method  C  performs  slightly  better 
than  HPSA  and  MUA  in  identifying  populations 
with  "structural"  or  access  problems. 

3 .  The  methods  developed  are  not  appropriate  for 
metropolitan  areas,  where  the  county  as  unit  of 
analysis  is  too  large. 


4.  Data  availability,  particularly  the  unit  of 
analysis  that  is  available,  is  a  large  factor  in 
determining  the  ability  to  make  changes  in  the 
present  means  of  designating  local  problem 
areas . 
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EFFECTS  OF  LIVING  IN  A  POOR  OR  UNDERCLASS 
NEIGHBORHOOD  ON  CANCER  SCREENING 

Janis  Barry  Figueroa,  Fordham  University 
Nancy  Breen 


Neighborhood-level  factors  may 
preclude  access^  to  health-care  resources 
or  the  ability  to  utilize  relevant  new 
medical  knowledge,  contributing  to  small- 
area  variations  in  health  outcomes.  We 
test  whether  four  residential  indicators 
explain  variations  in  diagnostic-stage 
among  women  with  cervical  and  breast 
cancer. 

Previous  studies  have  found  that 
socioeconomic  status  (broadly  defined) 
plays  a  significant  role  in  determining 
cervical  and  breast  cancer  screening, 
diagnosis  and  survival  (  Burg,  et.  al., 
1990;  Farley  and  Flannery,  1989;  Funch, 
1986) .  We  examine  whether  Census  tract 
measures  of  economically  and  socially 
declining  inner-city  areas  increase  a 
number  of  risk  factors  that  are 
associated  with  a  later-stage  diagnosis 
of  breast  and  cervical  cancer.  Our  study 
implicitly  considers  how  neighborhood- 
level  factors  lead  to  later-stage 
diagnosis  through  the  underutilization  of 
screening  services  by  medical  providers 
and/or  women  seeking  care.  We  stress 
neighborhood  attributes  that  both 
influence  individual  behavior  and 
undermine  the  capacity  of  institutions 
such  as  schools  or  health  clinics  to  be 
responsive  to  the  needs  of  the  community. 
Inner-city  neighborhoods  have  less  access 
to  markets.  We  would  expect  the  guality 
and  guantity  of  health  care  services  used 
in  these  areas  to  vary  according  to 
physician  advice  and  the  valuations  given 
to  preventive  care  by  community  residents 
(Barry  Figueora  and  Breen,  1995) . 

The  lack  of  routine  collection  of 
socioeconomic  data  on  medical  records, 
disease  registries  and  other  vital 
statistics  has  limited  the  ability  to 
identify  the  causal  process  through  which 
the  social  environment  influences  health 
outcomes  (Krieger,  1992) .  For  example, 
the  best  data  on  individual  cancer 
incidence  and  type,  is  available  from  the 
SEER  (Surveillance,  Epidemiology,  and  End 
Results)  program.  However,  it  does  not 
collect  data  on  individual  income, 
occupation,  education  and  health  care 
coverage.  Therefore,  we  use  Census  tract- 
characteristics  to  augment  SEER  data. 
RESIDENTIAL  LOCATION  AND  CANCER  DIAGNOSIS 
Rather  than  using  census-tract  data  as 
a  proxy  for  individual  attributes,  we  use 
area  data  to  demonstrate  the  importance 
of  neighborhood  characteristics.  The 
urban  underclass  literature  suggests  that 
cultural,  economic,  social  and 
technological  change  nave  connected  the 
fates  of  individuals  and  families  living 
in  the  inner-city  to  their  communities 
(Mincy,  1994) .  High-risk,  urban 
neighborhoods   characterized  by  vacant 


housing,  small-business  abandonment,  poor 
schools,  violent  crime,  illiteracy,  and 
disease  reflect,  in  part,  the  inadeguacy 
of  social  support  networks  in  the  face  of 
concentrated  and  persistent  poverty.  We 
assume  that  the  demand  for  health  care  is 
affected  by  residential  segregation  and 
isolation  from  mainstream  institutions  in 
neighborhoods  where  income  is  derived 
from  sources  other  than  work  in  the 
formal  economy  (Wolfe,  1994)  . 

The  "underclass"  and  "severely 
distressed"  neighborhood  indicators 
transcend  standard  disciplinary 
definitions.  They  include  both  geographic 
(spatial  concentration  of  poverty  in 
urban  areas) ,  economic  (employment  rates) 
and  social-cultural  factors  (rates  of 
unmarried  motherhood)  that  distinguish 
them  from  measures  of  either  income  or 
behavior  alone. 
DATA  AND  METHODS 

In  our  model  the  binomial  dependent 
variable  measures  the  likelihood  that 
women  are  diagnosed  after  the  tumor  is 
invasive  (l=invasive) .  We  analyze  cases 
of  breast  and  cervical  cancer  diagnosed 
between  1989  and  1990  that  could  be 
matched  with  Census-tract  data  in  the 
Atlanta,  Detroit  and  San  Francisco  SEER 
program  areas.1  These  three  cities  were 
chosen  because  they  are  major 
metropolitan  areas  with  both  high-  and 
low-income  Census  tracts  and  racially  and 
ethnically  diverse  populations. 
Independent  variables  include  individual 
measures  of  race  (non-Hispanic  white=0) , 
age,  marital  status  (married=0) ,  and  city 
of  residence  (San  Francisco=0) .  Key 
independent  variables  are  matched  Census- 
tract  indicators  of  residency  in  an 
underclass  (=1) ,  an  extremely  poor  (=1) , 
severely  distressed  (=1)  or  in  a 
medically  underserved  area  (=1) .  The 
underclass  measure,  developed  by  Ricketts 
and  Sawhill  (1988)  flags  a  Census  tract 
if  all  of  the  following  proportions  are 
at  least  one  standard  deviation  above  the 
national  mean:  (i)  16-19  year-olds  not 
enrolled  in  school  and  not  high  school 
graduates,  (ii)  males  aged  16  and  older 
out  of  work  for  more  than  26  weeks,  (iii) 
households  receiving  public  assistance 
income  and  (iv)  female-headed  households 
with  children.  Our  poverty  measure, 
developed  by  Jargowsky  and  Bane  (1991) , 
groups  Census  tracts  into  three  types: 
(i)  less  than  20  percent  of  households  in 
poverty,  (ii)  between  20  percent  and  40 
percent  in  poverty,  or  (iii)  more  than  40 
percent  in  poverty.  Severely  distressed 
neighborhoods  are  those  underclass  tracts 
that  in  addition,  register  at  least  one 
standard  deviation  above  the  national 
tract  mean  on  the  proportion  of  the 
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resident  population  below  the  poverty 
line  (Kasarda,  1993). 

The  federal  government  designates  an 
area  as  medically  underserved  (MUA)  when 
the  following  four-variable  index  exceeds 
a  specified  benchmark:  the  ratio  of 
primary  medical  care  physician-to- 
population,  the  infant  mortality  rate, 
the  percentage  of  the  population  with 
incomes  below  the  poverty  level,  and 
percentage  of  the  population  over  age  65 
(Code  of  Federal  Regulations,  1993  Ch  1 
[Appendix  C) )  . 

Using  logistic  regression  to  analyze 
each  cancer  site  separately,  we  estimate 
whether  residence  in  an  underclass, 
extremely  poor,  or  severely  distressed 
tract  is  significantly  associated  with 
later-stage  cancer  diagnosis.  We  expect 
that  women  who  live  in  these  areas 
will  be  more  likely  to  receive  later- 
stage  diagnosis  than  women  who  do  not, 
ceteris  paribus.  Furthermore,  we  expect 
that  the  underclass  and  severely 
distressed  measures  will  have  greater 
explanatory  power  than  the  pure,  income- 
based,  poverty  measure  in  predicting 
diagnostic  stage. 
RESULTS 

Approximately  19  percent  of  the 
cervical  cancer  tumors  and  87  percent  of 
the  breast  cancer  tumors  were  diagnosed 
as  invasive.  Three  percent  of  women 
diagnosed  with  later-stage  breast  cancer 
and  5  percent  with  cervical  cancer  lived 
in  underclass  areas  while  4  percent  and  6 
percent,  respectively,  lived  in  extremely 
poor  tracts.  Nine  percent  of  the  breast 
cancer  and  11  percent  of  the  cervical 
cancer  cases  lived  in  medically 
underserved  areas. 

Table  1  presents  the  variable 
proportions.  Table  2  gives  the  logistic 
point  estimates  for  the  breast  and 
cervical  cancer  models.  The  scale  factor 
given  in  the  first  row  is  the  density 
function  evaluated  at  the  means  of  the 
variables  (Green,  1994) .  In  brief,  the 
underclass  and  severely  distressed 
indicators  reveal  consistently  positive 
and  significant  effects  on  later-stage 
diagnostic  probabilities.  In  the  breast 
cancer  sample,  probabilities  of  later- 
stage  diagnosis  were  increased  by  4.3 
percent  and  4.4  percent,  respectively,  if 
the  individual  resided  in  an  underclass 
or  severely  distressed  neighborhood.  In 
the  cervical  cancer  sample,  the 
probabilities  of  later-stage  diagnosis 
were  increased  by  5.5  and  6.6  percent, 
respectively,  if  the  individual  lived  in 
an  underclass  or  severely  distressed 
area.  Residence  in  an  extremely  poor 
tract  was  a  significant  explanatory 
variable  only  in  the  cervical  cancer 
sample  increasing  later-stage  diagnostic 
probabilities  by  4 . 5  percent. 

Other  findings  for  the  breast  cancer 
sample  showed  that  the  probabilities  for 
later-stage  diagnosis  were  1.7  percent 
less  for  women  in  Atlanta  and  were  1.8 


percent  more  for  women  living  in  Detroit 
as  compared  to  women  in  San  Francisco. 
Living  without  a  spouse  and  yearly  age 
increases  elevated  later-stage  diagnostic 
probabilities  by  1.4  and  one-tenth  of  a 
percentage,  respectively. 

The  cervical  cancer  sample  revealed 

increased  probabilities  of  later-stage 

diagnosis  among  women  living  in  Atlanta 

(about  4%)  and  women  living  in  Detroit 

(about   3%).   Race   and  ethnicity  were 

significant  explanatory  variables  with 

black   non-Hispanics   facing   increased 

probabilities  of  3 . 5  percent  (columns  one 

and  three) ,  and  Hispanic  women  facing 

increased  probabilities  of  4.7  percent 

(columns   one   and   three) .   Age   was   a 

significant   predictor   of   diagnostic 

stage,  but  marital  status  was  not. 

DISCUSSION 

The  spatial  concentration  of  social 
disadvantage  as  measured  by  the 
underclass  and  severely  distressed 
indicators,  outperformed  the  income- 
based,  poverty  measure  in  explaining 
diagnostic  outcomes.  Social  (family 
instability  and  low  education  levels)  and 
economic  distress  (labor  force  detachment 
and  reliance  on  public  assistance)  at  the 
neighborhood-level  are  likely  to  interact 
in  ways  that  both  influence  and  constrain 
individual  investments  in  health  and  use 
of  the  health-care  system.  Moreover, 
health  care  markets  are  not  oriented 
toward  servicing  physically  isolated  and 
socially  disadvantaged  locales,  so  that 
opportunities  for  early  diagnosis  of 
these  cancers  are  lost. 

The  MUA  variable,  which  marks  a 
shortage  of  health-care  services  in  poor 
and  elderly  communities,  was  consistently 
insignificant.  This  may  be  due  partly  to 
the  fact  that  clinics  set  up  as  a  result 
of  the  MUA  designation  vary  considerably 
in  the  range  of  services  they  provide, 
especially  cancer  screening  and 
treatment.  The  poor  also  may  face 
barriers  to  timely  medical  care  that  are 
not  simply  a  matter  of  access  to  health- 
care services.  In  the  case  of  cervical 
cancer,  even  after  controlling  for 
neighborhood  effects,  the  city  of 
residence,  and  racial  and  ethnic 
identifiers  remained  important 
determinants  of  later-stage  diagnosis. 
This  suggests  the  need  for  continued 
research  on  special  populations  and 
specific  regions  of  the  United  States 
when  designing  cervical  cancer  prevention 
programs. 

The  study  of  neighborhoods  can 
motivate  a  broader  integration  and 
synthesis  of  the  literature  on 
socioeconomic  status  and  health.  It  can 
also  challenge  the  existing  research  by 
raising  questions  about  the  links  between 
the  financial  and  behavioral  explanations 
for  health  outcome  inequalities.  Further 
investigation  of  why  individuals  do  not 
get  the  preventive  care  they  need  awaits 
the  collection  of  individual-level  social 
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and  economic  data  on  medical  data  bases. 

Systematic   differences   between   the 

average  health  status  of  people  by  region 

or  geographic  locale  signal  that  space 

and  place  information  should  be  included 

when  analyzing  individual  determinants  of 

health. 
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Table  1 

SAMPLE  PROPORTIONS  OF  EXPLANATORY  VARIABLES 

Breast  Cancer       Cervical  Cancer 

Independent  Variable 

.185 

.197 

.457 

.194 

.060 

.025 

37.5 

.559 

.047 

.194 

.062 

.043 

.114 

SOURCE:  1989-1990  SEER  reporting  areas  for  Atlanta,  Detroit  and  San 
Francisco  and  1990  Census  Summary  Tape  Files  3A. 


Later-Stage  Diagnosis 

.871 

Atlanta 

.183 

Detroit 

.424 

Black/NH 

.148 

Hispanic 

.026 

Asian/PI 

.032 

Age  (mean) 

61.1 

UnMarried 

.465 

Underclass 

.029 

Poverty<4  0% 

.143 

Poverty>40% 

.043 

ServDistres 

.026 

M.U.A. 

.086 
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Table  2    Logit  Estimates  of  Stage  of  Diagnosis  Equations, 
Breast  and  Cervical  Cancer 


Breast 


Cervix 


Independent 
Variables    (1) 


(2) 


(3) 


(1) 


(2) 


(3) 


dP(Y=l) 
dXt  x 

.1110 

.1111 

.1096 

.1218 

.1215 

.1220 

Atlanta 

-.156* 
(2.18) 

-.160* 
(2.23) 

-.158* 
(2.20) 

.346** 
(2.89) 

.326* 
(2.72) 

.341** 
(2.85) 

Detroit 

.171* 
(2.78) 

.169* 
(2.72) 

.169* 
(2.76) 

.279* 
(2.77) 

.245* 
(2.39) 

.267* 
(2.64) 

Black/NH 

.089 
(1.08) 

.069 
(.763) 

.092 
(1.12) 

.295** 
(2.85) 

.210* 
(1.84) 

.291** 
(2.82) 

Hispanic 

.168 
(1.00) 

.167 
(1.00) 

.171 
(1.02) 

.393* 
(2.23) 

.365* 
(2.05) 

.389* 
(2.20) 

Asian/PI 

-.011 
(.076) 

-.015 
(.105) 

-.012 
(.086) 

.245 
(1.05) 

.222 
(.956) 

.240 
(1.03) 

Age 

.015** 
(8.01) 

.015** 
(8.01) 

.015** 
(8.01) 

.074** 
(28.1) 

.074** 
(28.2) 

.074** 
(28.5) 

UnMarried 

128* 
(2.30) 

.127* 
(2.28) 

.128* 
(2.30) 

-.018 
(.229) 

-.028 
(.349) 

-.021 
(.255) 

Underclass 

.394* 
(1.99) 

- 

- 

.455* 
(2.60) 

- 

- 

Poverty<4  0% 

- 

.087 
(.811) 

- 

- 

.150 
(1.14) 

- 

Poverty>40% 

- 

.085 
(.502) 

- 

- 

.376* 
(2.13) 

- 

ServDistres 

- 

- 

.410* 
(1.93) 

- 

.541** 
(3.02) 

M.U.A. 

.096 
(.910) 

.075 
(.670) 

.100 
(.956) 

.071 
(.567) 

-.022 
(.349) 

.073 
(.589) 

Constant 

-2  X  Log 
Likelihood 

.867** 
(7.24) 

10157 

.869** 
(7.24) 

10157 

.867** 
(7.24) 

10157 

-4.88** 
(32.8) 

4116 

-4.86** 
(32.6) 

4116 

-4.87** 
(32.7) 

4116 

N  of  Cases 

13398 

5401 

Multiply  this  value  times  the  coefficent  estimate  to  get  the  marginal 
contribution  of  each  explanatory  variable  to  the  probability  of  late- 
stage  diagnosis. 

NOTE:  Dependent  Variable:  Probability  of  individual  woman  being 
diagnosed  with  late-stage  breast  or  cervical  cancer.  T-ratios  are  in 
parentheses.  *p  <  .05;  **p  <  .01. 

SOURCE:  1989-91  SEER  reporting  areas  for  Atlanta,  Detroit  and  San 
Francisco  and  1990  Census  Summary  Tape  File  3A. 
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Introduction 

As  scientific  evidence  about  the 
effects  of  exposure  mounts,  low 
childhood  lead  levels  have  been 
reclassified  as  toxic'   The  Centers  for 
Disease  Control  and  Prevention  considers 
lead  levels  greater  than  10  micrograms 
per  deciliter  unacceptable.2  Although 
some  states  require  screening  of  all 
children  under  6  years  of  age, 
controversy  over  the  need  for  universal 
screening  has  erupted.3,4 

Arguments  against  universal 
screening  include  low  prevalence  of  lead 
poisoning,  lack  of  efficacious  treatment 
for  slightly  elevated  lead  levels,  and 
false  positive  rates  for  capillary 
samples. J-5  However,  low  prevalence 
rates  represent  large  numbers  of 
children  effected  by  plumbism,  and 
population-based  prevalence  studies  are 
rare.6  Efficacy  of  treatment  for 
elevated  lead  levels  has  been 
established  and  sensitivity  and 
specificity  of  pediatric  lead  testing 


has  been  evaluated.' 


Local  communities 


lack  data  for  targeting. 

Epidemiological  research  offers 
evidence  for  targeting.   Children  at 
greater  risk  include  inner  city 
residents,  the  poor,  minorities,  and 
children  residing  in  deteriorated 
housing.910  The  risk  of  lead  poisoning 
in  older  dwellings  has  been 
reiterated.11"17  Cincinnati  researchers 
found  moderate  correlations  between 
blood  lead  levels  and  housing  age.18 
Lead  levels  in  6 -month-old  children 
differed  by  housing  categories,  with 
levels  highest  in  pre-World  War  II 
deteriorated  housing.19  Housing  age  has 
also  been  strongly  associated  with 
breast  milk  lead.20 

Despite  this  evidence,  measurement 
and  analysis  of  the  relationship  between 
lead  and  housing  is  often  flawed. 
Categorization  of  year  of  structure  is 
misleading  and  may  influence  variance 
estimation.   The  age,  type  and  quality 
of  a  community's  housing  stock  may 
differ  within  the  same  region. 
Interaction  effects  either  have  not  been 
investigated  or  have  not  been  reported.21 
Since  geometric  means  are  lower  than 
arithmetic  means,  the  latter  over 
estimates  exposure.   The  distribution  of 
lead  is  non-gaussian,  and  failure  to  use 
data  transformations  underestimates 
variance.   As  Dabeka  and  the  World 
Health  Organization  have  recommended, 
geometric  means  are  a  preferred  method 
to  represent  lead  data.20-22 


For  targeting  lead  screening 
activities  in  Mahoning  County,  Ohio,  a 
population  based  prevalence  study  was 
conducted  to  determine  the  association 
of  housing  age  and  condition  with 
childhood  blood  lead  levels. 

Methodology 

Subjects 

To  obtain  age  of  dwellings,  birth 
certificate  addresses  of  children 
residing  in  Mahoning  County  were  matched 
to  county  auditor  data  for  births 
between  February  1992  and  February  1993 . 
Mothers  included  in  the  study  were  those 
living  continuously  in  the  same  house 
since  the  birth  of  the  study  child. 
Initial  phone  contact  using  a  reverse  or 
city  directory  began  in  January  1994 . 
Households  which  were  not  listed  were 
contacted  by  mail  on  at  least  two 
attempts.   Mail  carriers  also  identified 
whether  the  name  and  address  on  the 
birth  certificate  matched  with  the 
occupant  of  the  dwelling. 
Data   collection  and  analysis 

Teams  of  phlebotomists  and  survey 
interviewers  were  trained  regarding 
sample  collection  and  survey  response. 
During  a  scheduled  visit  to  the  home, 
the  phlebotomist  explained  the  study, 
obtained  informed  consent  and  drew  a 
blood  sample  of  the  child.   The  survey 
interviewer  conducted  a  maternal 
questionnaire  and  a  visual  assessment  of 
the  home . 

Venipuncture  samples  were  obtained 
on  60  percent  of  study  children,  and 
capillary  samples  were  obtained  for  the 
remainder.   Approximately  25  mothers 
indicated  previous  lead  testing  for 
their  child.   The  interviewer 
assessment,  maternal  survey  and  release 
of  medical  information  forms  were 
completed  and  lead  levels  were  obtained 
from  the  pediatrician. 

Blood  analysis  was  performed  by  a 
private  certified  lab  using  atomic 
absorption  spectrophotometry,  with  a 
detection  limit  of  1  microgram  per 
deciliter.   Results  were  reported  to  the 
Mahoning  County  Childhood  Lead  Poisoning 
Prevention  Program  and  personal 
pediatricians  for  appropriate  follow-up. 
Eligibility  and  Representativeness 

Only  3  6  percent  of  birth 
certificate  identified  children  were 
eligible  to  participate  in  the  study. 
Results  are  shown  in  Table  1.   Based 
upon  information  on  mail  delivery  from 
postal  carriers,  "not  known"  respondents 
were  classified  as  not  eligible  to 
participate  in  the  study.   As  suggested 
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by  the  comparison  of  several  study- 
variables,  such  as  year  of  structure, 
maternal  age,  years  of  education, 
birthweight  in  grams,  and  proportion 
white,  such  a  procedure  seemed 
reasonable.   Average  maternal  years  of 
completed  school  for  the  not  eligible 
mothers,  12.2  years,  was  similar  to  the 
average  education  level  of  the  "not 
known"  mothers,  12.4  years.   Eligible 
household  mothers  had  completed  an 
average  of  13.4  years  of  education. 

Response  rate  among  eligible 
caregivers  was  almost  50  percent.   To 
assure  sample  representativeness, 
mothers  who  refused  to  participate  in 
the  study  were  compared  to  mothers 
participating  in  the  study.   See  Table 
2.   Maternal  educational  level  and  age 
for  respondents  and  nonrespondants  were 
similar.   Although  mothers  with  more 
recent  homes  were  slightly  more  likely 
to  refuse  to  participate  in  the  study  as 
were  white  mothers,  no  substantive 
differences  between  the  groups  were 
detected.   Average  year  of  dwelling 
construction  for  mothers  refusing  study 
participation  was  1955  and  average  year 
of  structure  for  mothers  entering  the 
study  was  1949 .   Over  90  percent  of 
nonrespondents  were  white,  compared  to 
86  percent  of  study  participants. 
Variables  and  data  analysis 

Geometric  or  arithmetic  means  were 
calculated  and  compared  to  NHANES  III 
data.23,24  Variables  included  in  the 
analysis  were  blood  lead  levels  in 
micrograms  per  deciliter,  a  rating  by 
the  occupant  of  the  dwelling  as 
"excellent",  "good",  "fair"  or  "poor" 
(coded  one  through  four,  with  four 
representing  "poor"  housing)  and 
ethnicity  (dummy  coded  white  and 
nonwhite  with  nonwhite  equal  to  1) . 
Actual  year  of  dwelling  construction  was 
used  in  the  regression  analysis. 
Samples  below  the  limit  of  detection 
were  assigned  the  value  of  one-half  of 
the  detection  limit.   Because  lead 
levels  were  not  normally  distributed,  a 
data  transformation,  the  base  ten 
logarithm,  was  applied  to  the  variable 
before  predictive  regression  analysis. 

Findings 

Distribution  of  blood  lead 

Geometric  mean  blood  levels  for 
children  in  Mahoning  County  were  similar 
to  means  reported  for  NHANES  III  study 
children.   See  Figure  1.   For  NHANES 
children  aged  1  to  2 ,  the  mean  blood 
lead  level  was  4 . 1  micrograms  per 
deciliter  and  for  children  aged  3  to  5 
the  mean  was  3.4  micrograms  per 
deciliter.   For  Mahoning  County  children 
aged  1  to  3,  the  mean  blood  lead  level 
was  3.3  micrograms  per  deciliter. 

The  distribution  of  blood  lead 
levels  in  Mahoning  County  and  in  the 


NHANES  study  was  similar.   In  Mahoning 
County,  29.6  percent  of  children  1  to  3 
years  old  exhibited  blood  lead  levels 
greater  than  5  micrograms  per  deciliter, 
the  median.   In  the  NHANES  study,  28.6 
percent  of  the  children  exhibited 
similar  blood  lead  levels.   Because  of 
confidence  interval  differences 
attributed  to  differences  in  study 
methodologies,  no  statistical  tests  were 
performed. 
Bivariate  analysis 

As  found  in  the  NHANES,  geometric 
mean  blood  lead  levels  for  Mahoning 
County  nonwhite  children  were  higher 
than  mean  levels  for  white  children. 
See  Figure  3 .   The  geometric  mean  blood 
lead  level  for  non-hispanic,  white 
children  aged  1  to  3  was  3 . 0  micrograms 
per  deciliter,  and  similar  to  the 
geometric  mean  for  such  children 
reported  in  the  NHANES,  2.9  micrograms 
per  deciliter.   For  non-hispanic  blacks, 
the  geometric  mean  blood  lead  for 
Mahoning  County,  6.6  micrograms  per 
deciliter,  was  twice  as  great  as  the 
mean  lead  level  for  white  children. 

Blood  lead  levels  were  highest  for 
structures  built  prior  to  1940  and  were 
lowest  for  dwellings  built  after  1980. 
See  Figure  4 .   Arithmetic  mean  blood 
lead  was  6.82  micrograms  per  deciliter 
in  pre  1940  dwellings  and  decreased 
monotonically  for  more  recently  built 
dwellings.   Arithmetic  blood  lead  mean 
for  post  1980  dwellings  was  only  2.48 
micrograms  per  deciliter. 

Blood  lead  levels  were  highest  for 
dwellings  which  mothers  rated  as  "poor" 
and  lowest  for  dwellings  which  mothers 
characterized  as  "excellent".   Se3 
Figure  5.   Blood  lead  levels  were  2.83 
micrograms  per  deciliter  among  dwellings 
rated  as  "excellent"  and  were  7.82 
micrograms  per  deciliter  for  homes 
described  as  "poor" . 
Regression  analysis 

Blood  lead  levels  were  analyzed 
using  multiple  regression  with  forced 
entry  of  the  variables  race,  maternal 
rating  of  dwelling,  and  year  of 
structure.   Thirty  percent  of  the 
variability  in  blood  lead  for  study 
children  was  explained  by  housing  age, 
condition,  and  maternal  ethnicity.   See 
Table  3 .   No  interaction  terms  among  the 
variables  were  noted.   Standardized  and 
unstandardized  regression  coefficients 
and  statistical  tests  for  whether  or  not 
terms  in  the  model  differed  from  zero 
are  shown  in  the  table. 
Spatial   analysis 

Program  data  from  the  Mahoning 
County  lead  program,  study  data  and  a 
map  for  Youngstown,  Ohio  generated  using 
TIGER,  a  geographic  information  system 
available  from  the  Census  Bureau  are 
shown  in  Figure  6 .   Three  neighborhoods 
located  north,  east  and  south  of  the 
central  business  district  of  Youngstown 
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had  a  preponderance  of  exposed  children. 
Census  information  suggested  these 
neighborhoods  are  predominantly  nonwhite 
with  older,  poorer  quality  housing 
stock. 

Discussion 

In  Mahoning  County,  age  and 
condition  of  housing  stock  was 
associated  with  childhood  blood  lead 
levels.  Inner  city  neighborhoods  were  at 
greater  risk  of  childhood  plumbism. 
These  neighborhood  characteristics  could 
be  used  to  target  screening  activities. 
Mahoning  County's  model  of  blood  lead 
predicted  as  much  of  the  variability  in 
blood  lead  as  the  NHANES  model  with  far 
fewer  variables. 

In  a  recent  report,  2  7  percent  of 
the  variability  in  blood  lead  was 
explained  by  age,  gender,  race,  poverty, 
level  of  education,  urban  and  region.23 
In  Mahoning  County,  3  0  percent  of  the 
variability  in  blood  lead  was  predicted 
using  year  of  structure,  race,  and 
rating  the  home  as  "excellent",  "good", 
"fair"  or  "poor",  thus  achieving  a  more 
parsimonious  predictive  model.   More 
research  is  needed  to  increase  the 
predictive  nature  of  the  model . 

Other  communities  could  use  this 
approach  to  target  screening  activities. 
Existing  data  to  target  and  evaluate 
screening  includes  programmatic  data, 
such  as  STELLAR,  a  case  management  and 
reporting  software  used  by  Ohio  lead 
programs.   TIGER  files  (aka  GBF/  DIME 
files  or  a  census  tract  street  index) 
could  also  be  used  to  target 
neighborhoods,  as  could  other  geographic 
or  census  information.   Urban  planning 
departments  or  local  land  use  experts 
may  also  have  information  which  would  be 
useful  for  targeting  screening  efforts. 
However,  such  efforts  should  be  guided 
by  knowledge  of  the  bias  in  screening 
data.   Screening  data  may  not  reflect 
true  population-based  prevalence  rates. 

Mahoning  County  auditor  data  was 
particularly  helpful  in  obtaining  study 
data .   Tax  records  kept  by  Mahoning 
County  identified  structures,  their 
approximate  age,  renovations  made  to 
structures,  the  quality  of  neighborhoods 
in  which  a  dwelling  is  located,  recent 
market  value  of  buildings,  and  many 
other  variables  which  could  be  useful  in 
targeting  screening.   Tax  records 
represent  an  objective,  independent  and 
unobtrusive  means  of  collecting  data  on 
housing  units. 

While  some  epidemiologists  have 
attributed  lead  exposure  to  hand-to- 
mouth  activity  and  pica  of  children, 
others  have  suggested  that  lead  dust 
contributes  to  blood  lead  levels.   The 
controversy  over  inhalation  and 
ingestion  of  lead  as  biological  pathways 
to  exposure  is  not  resolved  by  this 


data.   Yet,  the  variables  housing  age 
and  condition  may  represent  a  proxy 
variable  which  measures  both  pathways 
and  is  adequate  for  targeting. 

To  shed  light  on  biological 
pathways  the  Cincinnati  Study  has 
investigated  the  relative  contribution 
of  dust  and  hand  lead.  Explaining  how 
socioeconomic  status  directly  effects 
blood  lead  levels  remains  problematic. 
Explanations  for  the  racial  disparity  in 
blood  levels  and  whether  race 
differences  have  been  exacerbated  over 
time . 

Changes  in  the  relationship  between 
housing  age  and  condition  and  blood  lead 
over  time  warrant  further  investigation. 
Monitoring  such  changes  is  necessary  for 
targeting  screening  and  perennial 
adjustments  to  include  more  or  different 
neighborhoods.  As  a  community's  housing 
stock  ages,  pre  1978  dwellings  which 
were  not  "lead  free"  but  "lead  safe"  may 
represent  a  threat . 

According  the  1990  Census,  79 
percent  of  the  housing  units  in  the 
Unites  States  were  built  prior  to  1980. 2i 
Lead  will  continue  to  be  a  persistent 
poison.   Children  at  greatest  risk  can 
be  identified  for  targeted  screening 
activities  with  data  such  as  that  used 
in  Mahoning  County . 
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ELIGIBILITY 

Table  1 


NOT 
ELIGIBLE 

ELIGIBLE 

NOT 
KNOWN 

YEAR  HOME  BUILT 

1839  ±  104 

1952  t  26 

1940  ±  26 

MATERNAL  AGE,  y«n 

24.4  s  S.9 

28.9  ±  5.8 

25.6  ±  6.5 

EDUCATION 

12.2  t  2.1 

13.4  1  2.1 

12.4  ±  1.9 

(Yaar*  Completed) 

BIRTHWEIGHT,  gum 

3206  ±  670 

3367  ±  580 

3288  I  570 

%  WHITE 

55.5  % 

66.1  % 

66.9  % 

NOT  EUQIBIE:  EWIREO  N*».  MOVED  N»ia>6.  OTHER  N«3 
ELIGIBLE;  REFUSED  NM41,  COMPU6TEO  N-421  (36  %  OF  BIRTHS) 
NO  CONTACT  IMPLIED  REFUSED  N«4*T.  DISCONNECTED  N.J.  WRO 


RESPONSE  BIAS 


Table  2 


OVERALL 

COMPLETED 

REFUSED 

TEST 

P 
VALUE 

YEAR  HOME  BUILT 

1962   X    28 

1»49  ±    26 

1966    ±    26 

Foil. 21  6 

0.001 

MATERNAL  AGE.  f — 

28.8    ±    6  0 

26.9    i    6   a 

F=0.16fi 

0.667 

EDUCATION 

13.41   *  2.1 

13.4   ±   1.1 

13.6  ±   2.1 

F  =  0.442 

0.608 

pmn  co«npi*Ud) 

B1RTHWEIGHT.  grams 

3367   I   660 

3366  ±    676 

33/6  ±   666 

F  =  0.234 

0.629 

X  WHITE 

.*,  * 

66.9  % 

K.2S 

chl2o3.607 

0.061 

REFUSED  N-441,  COMPLETED  N-421;  RESPONSE  RATE  48  8  % 
OF  THOSE  ELIGIBLE    36  %  OF  BIRTHS 


Geometric  Mean  Blood  Lead 
microgram/dL 


NHANES  III  ago  1-2  NHANES  III  age  3-5  Young8lov,n  ago  1-3 


ages  1-2 
ages  3-5 

Figure 
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Distribution  of  Blood  Lead 
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Geometric  Mean  Blood  Lead 
By  Ethnicity 
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by  Housing  Age 
Arithmetic  Mean 


<   1940         1940-1959      1960-1979         1980   > 
Housing  age 

Figure  4 

Blood  Lead  Levels 

by  Maternal  Rating  of  Home 

Arithmetic  Mean 


10 
-I     8 

■D 

l      6 

oS 

£     4 
jo 

E      2 
0 


7.82- 


Excellent  Good  Fair  Poor 

Maternal  Rating  of  Home 


Figure  5 


Linear  Regression  Analysis 
log  10  blood  lead  level,  Youngstown,  Ohio 

Table  3 


Variable 

Unstandardized 
Coetfloient 

Standardized 
Coefficient  6 

students 
ttest 

P 
value 

Year 

-0.0036 

-0.3409 

•7.153 

<0.0001 

Race 

0.2157 

0.2566 

5.768 

<0.0001 

Home 

0.0563 

0.1400 

3.034 

0.0026 

Constant 

7.6815 

7.415 

<0.0001 

Model  Multiple  R=0.553,  R2  =  0.306,  Adjusted  R2=0.300 
Model  F  56.93,  p  <  0.0001 
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IMPLICATIONS  OF  ALTERNATIVE  DEFINITIONS  OF  DISABILITY  IN  CHILDREN 

Lauren  E.  Westbrook,  Albert  Einstein  College  of  Medicine,  Bronx,  NY 
Ellen  J.  Silver  and  Ruth  E.K.  Stein 


INTRODUCTION 

Data  on  children  with  disabilities 
are  a  priority  right  now  for  research, 
policy  and  programmatic  purposes. 
Significant  budget  cuts  to  Medicaid, 
widespread  implementation  of  managed 
care  programs  and  other  changes  in  the 
health  care  system  have  intensified  the 
need  to  plan  for  services  and  benefits 
to  children  with  chronic  and  disabling 
conditions. 

Among  the  key  issues  that  need  to 
be  addressed  is  how  to  determine 
eligibility  for  research,  programs  and 
services.   Therefore,  how  disability 
should  be  defined  for  children  is 
receiving  a  great  deal  of  attention. 
Checklists  of  disease  entities  or 
symptoms  to  identify  children  are  no 
longer  consistent  with  the  public  policy 
movement.   Rather,  there  has  been  a 
conceptual  shift  toward  a  more 
noncategorical  approach  (Pless  & 
Pinkerton,  1975;  Stein  &  Jessop,  1982). 
The  noncategorical  approach  to 
identifying  children  with  chronic 
conditions  and  disabilities  applies 
generic  criteria  by  examining  the 
consequences  of  diverse  health 
conditions  (Stein  et  al.,  1993). 

There  is  no  universally  accepted 
definition  of  disability.   Inconsis- 
tencies in  how  disability  is  defined 
make  interpretation  of  data  extremely 
difficult.   We  need  to  understand  the 
implications  of  using  different 
definitions  for  children  with  disabil- 
ities before  data  can  be  appropriately 
applied  to  policy  and  funding  decisions. 

As  part  of  a  larger  effort  to  begin 
to  elucidate  the  conceptual,  method- 
ological and  definitional  issues  around 
disability,  the  Office  of  the  Assistant 
Secretary  for  Planning  and  Evaluation 
has  funded  several  projects  to  conduct 
secondary  analyses  of  existing  disabil- 
ity data.   The  data  presented  here  are 
from  one  of  these  projects. 

The  purpose  of  our  study  is  to 
examine  the  consequences  and  health 
policy  implications  of  using  different 
components  of  a  noncategorically-based 
definition  of  disability  developed  by 
our  research  team  to  identify  children 
in  three  existing  data  sets.   This  study 
is  ongoing  and  we  have  not  yet  completed 
our  analysis,  but  do  have  some 
preliminary  results.   First,  the 
conceptual  definition  we  used  and  the 
tool  we  developed  to  operationalize  the 
definition  will  be  described.   Second, 
the  data  sets  utilized  for  this  analysis 
will  be  detailed.   Third,  our  findings 
to  date  will  be  presented  and  our  future 


analysis  plans  briefly  outlined. 
Finally,  we  will  offer  several 
conclusions  based  on  these  preliminary 
results. 

A  NONCATEGORICAL  QUESTIONNAIRE 

In  an  earlier  study,  also  headed  by 
Dr.  Ruth  Stein,  we  developed  a 
noncategorical  questionnaire  that 
reflects  the  consequences  and 
impairments  associated  with  chronic  and 
disabling  conditions  in  children  under 
18  years  of  age  (Stein  et  al.,  1993). 
The  questionnaire  taps  three  conceptual 
domains:  functional  limitations,  service 
use  beyond  routine  and  reliance  on 
compensatory  mechanisms  or  assistance. 
These  domains  represent  what  we  and  a 
national  advisory  panel  believe  to  be 
essential  components  to  a  comprehensive 
definition  of  disability  in  children 
that  relies  on  noncategorical  criteria. 
Each  of  these  definitional  components 
can  be  explained  as  follows. 

Functional  limitations  are 
conceptualized  as  limitation  of 
function,  activities  or  social  role  in 
comparison  with  healthy  age  peers  in  the 
general  areas  of  physical,  cognitive, 
emotional  and  social  growth  and 
development.   We  operational ized 
functional  limitations  as  being  limited 
in  play  with  other  children;  being 
restricted  in  activities;  having 
difficulty  feeding,  dressing,  washing  or 
toileting;  and  having  difficulty 
hearing,  seeing  or  communicating. 

Service  use  beyond  routine  care  is 
conceptualized  as  the  use  of  or  need  for 
medical  care  or  related  services, 
psychological  services  or  educational 
services  over  and  above  the  usual  for 
the  child's  age,  or  use  of  or  the  need 
for  special  ongoing  treatments, 
interventions  or  accommodations  at  home 
or  in  school.   This  domain  reflects 
current  service  use  as  well  as  expressed 
needs  for  services  that  are  not 
presently  being  met.   We  operationalized 
service  use  beyond  routine  care  as 
hospitalizations,  doctor  visits,  nursing 
care  or  treatment,  physical  or  speech 
therapy,  psychological  services  and 
special  arrangements  at  school  (for 
example,  classroom  or  schedule 
modifications) ,  special  instruction  or 
other  educational  services. 

Reliance  on  compensatory  mechanisms 
or  assistance  is  conceptualized  as 
dependency  on  an  accommodating  mechanism 
to  compensate  for  or  minimize  limitation 
of  function,  activities  or  social  role. 
Reliance  on  compensatory  mechanisms  or 
assistance  was  operationalized  as 
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medication  use,  special  diet  and  use  of 
special  equipment  such  as  assistive 
devices,  medical  technology  or  personal 
assistance. 

Functional  limitations  and 
utilization  of  medical  and  other  related 
services  typically  are  used  in  measures 
to  identify  and  describe  children  with 
disabilities.   The  concept  of  "compen- 
sated" function  with  accommodations, 
however,  is  relatively  new  (Verbrugge, 
1990)  and  has  not  before  been  applied  to 
children.   When  abilities  are  maximized 
and  symptoms  are  successfully  reduced, 
many  children  who  have  disabilities  will 
be  functionally  indistinguishable  from 
healthy  peers.   But  in  order  for 
successful  compensation  to  occur,  other 
consequences  are  necessarily  imposed  on 
these  children.   These  compensatory 
consequences  are  what  we  are  tapping  in 
this  domain.   For  example,  the  child 
with  epilepsy  whose  seizures  are 
controlled  with  medication  would  be 
identified  with  the  item  on  medication 
use.   The  child  with  phenylketonuria  who 
can  live  a  normal  life  as  long  as  a 
special  diet  is  rigorously  maintained 
would  be  identified  with  the  item  on 
special  diet.   In  both  instances, 
absence  of  treatment  would  be  expected 
to  produce  functional  impairments. 

The  questionnaire  we  developed, 
called  the  Questionnaire  for  Identifying 
Children  with  Chronic  Conditions 
(QuICCC) ,  identifies  children  who  are 
currently  experiencing  consequences  in 
any  of  the  three  domains  just  described: 
functional  limitations,  service  use 
beyond  routine  and  reliance  on 
compensatory  mechanisms  or  assistance. 
The  QuICCC  was  originally  designed  to  be 
used  as  a  household  survey.   The 
respondent  is  usually  the  parent  or 
caregiver  who  is  the  most  knowledgeable 
about  the  health  of  the  children  living 
in  the  household.   Because  the  age  range 
is  so  broad  (from  birth  to  age  18)  some 
of  the  questions  are  age  specific. 

The  QuICCC  consists  of  39  items. 
The  items  are  generally  structured  in 
three  parts,  as  shown  by  the  example 
provided  in  Table  1.   A  child  is 
identified  as  having  a  chronic  or 
disabling  condition  only  if  affirmative 
responses  are  given  to  all  parts  of  a 
question  sequence.   Table  1  shows  the 
item  on  medication  use  which  reflects 
the  domain  of  reliance  on  compensatory 
accommodations.   The  respondent  is  first 
asked  if  any  of  the  children  take 
medicine  or  drugs  prescribed  by  a  doctor 
other  than  regular  vitamins.   If  the 
respondent  answers  "No"  or  "Don't  Know", 
the  interviewer  skips  to  the  next 
question.   If  the  respondent  answers 
"Yes",  the  interviewer  asks  if  this  is 
because  of  a  medical,  behavioral  or 
other  health  condition  that  the  child 
still  has?   If  the  answer  is  "No"  or 
"Don't  Know",  the  interviewer  skips  to 


TABLE  1.  SAMPLE  ITEM  FROM  THE  QUICCC 

la.   Do  any  of  the  children  [in  the 
household]  take  medicine  or 
drugs  prescribed  by  a  doctor 
other  than  regular  vitamins? 

1  (  )  Yes 

2  (  )  No 

3  (  )  Don't  Know 

b.  Is  this  because  of  a  medical, 
behavioral  or  other  health 
condition  that  the  child  still 
has? 

1  (  )  Yes 

2  (  )  No 

3  (  )  Don't  Know 

c.  Has  this  condition  been  going 
on  or  is  it  expected  to  go  on 
for  at  least  one  year? 

1  (  )  Yes 

2  (  )  No 

3  (  )  Don't  Know 


the  next  question.   If  the  answer  is 
"Yes",  the  interviewer  asks  the 
respondent  if  this  condition  has  been 
going  on,  or  is  expected  to  go  on,  for 
at  least  one  year.   Only  if  the 
respondent  answers  "Yes"  to  this  and 
each  proceeding  part  of  the  question  is 
the  child  (or  children)  identified. 

An  additional  question  at  the  end 
of  each  item  that  asks  for  the  name  of 
the  condition  is  optional  with  the 
QuICCC.   We  included  this  query  in  our 
field  tests  for  additional  information, 
but  it  is  neither  essential  to  our 
definition,  nor  does  it  affect  the 
QuICCC 's  ability  to  identify  children. 

RESEARCH  QUESTIONS 

The  39  item  sets  of  the  QuICCC  were 
divided  to  reflect  each  of  the  three 
components  of  our  noncategorical 
definition  so  that  we  could  compare  the 
consequences  and  health  policy 
implications  of  using  the  individual 
components  separately. 

The  analyses  that  follow  address 
two  specific  research  questions.   First, 
what  proportions  of  children  are 
identified  as  disabled  using  different 
components  of  our  noncategorical 
definition?   Second,  are  children  with 
different  diagnoses  and  conditions 
identified  using  different  components  of 
our  definition? 

SAMPLE  CHARACTERISTICS  AND  METHODS 

Data  from  the  QuICCC  are  currently 
available  from  two  different  sample 
populations:  the  National  Sample  is  a 
cross-sectional  sample  of  1388  children 
from  a  random-digit-dial  telephone 
survey  of  households  conducted  across 
the  United  States,  and  the  Inner-City 
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Sample  is  a  cross-sectional  sample  of 
1275  predominantly  minority  children 
from  a  random-digit-dial  telephone 
survey  of  households  in  a  Northeastern 
inner-city.   Table  2  describes  some  of 
the  characteristics  of  the  samples. 


were  identified  by  service  use. 
However,  one  also  can  see  that  if  we  had 
limited  our  definition  to  service  use  as 
we  operationalized  it,  or  to  any  other 
single  component  for  that  matter,  a 
significant  proportion  of  children  would 
have  been  mis-classified  as  healthy. 


TABLE  2.   SAMPLE 

CHARACTERISTICS 

National 
N 

children       1388 
households      712 

Inner-City 

1275 
657 

Sex  (%) 

male 
female 

50 
50 

50 
50 

Age  (%) 

mean  (yrs) 
0-5  years 
6-10  years 
11-17  years 

8.2 

34 
30 
36 

7.6 
41 
27 
32 

Race/Ethnicity  ( 
white 
non-white 

*) 

80 
20 

14* 
86 

Education  (%) 
<  high  school 
h.s.  grad 
>  high  school 

11 
38 
51 

29* 

29 

42 

Income  (%) 
<  $15,000 
>  $15,000 

17 
83 

38* 
62 

*  p  <.001 

RESULTS 

The  proportions  of  children 
identified  by  each  of  the  three 
components  of  the  definition  are  shown 
in  the  following  figures.   Figure  1 
shows  that  in  the  National  Sample,  258 
of  the  1388  children  sampled  or  19%  were 
identified  as  disabled  using  the  QuICCC. 
Slightly  less  than  half  (44%)  met 
criteria  in  only  one  component  of  the 
definition.   More  than  one-third  (36%) 
met  two  of  the  three  definition 
components  and  20%  were  identified  by 
items  in  all  three  components. 

We  wanted  to  examine  more  closely 
those  children  who  were  identified  by 
only  one  component  of  the  definition. 
Figure  1  also  shows  a  further  breakdown 
of  this  group.   We  found  that  30%  were 
identified  by  reliance  on  compensatory 
mechanisms  or  assistance,  47%  by  the 
service  use  component  and  2  3%  were 
identified  as  having  functional 
limitations.   The  majority  of  children 


NATIONAL  SAMPLE: 
PROPORTION  IDENTIFIED  AS  DISABLED 


3   COMPONENTS 


9  SERVICE   USE 


TOTAL  SAMPLE  IDENTIFIED  AS  DISABLED 
268/1388  (19%) 


Figure  1, 


Proportion  Identified  as 
Disabled  in  National  Sample 


We  repeated  this  analysis  in  the 
Inner-City  Sample  and  found  quite 
similar  results.   Figure  2  shows  that 
overall,  246  out  of  the  total  sample  of 
1275  children  (19%)  were  identified  as 
disabled  by  at  least  one  component  of 
the  Child  Health  Screen.   This  is 
exactly  the  same  proportion  of  children 
as  identified  in  the  National  Sample. 
Slightly  more  than  half  of  these 
children  (52%)  met  the  criteria  for  only 
one  component,  32%  met  two  components  of 
the  definition  and  16%  met  all  three 
components . 

Figure  2  also  depicts  the  breakdown 
of  children  identified  by  only  one 
component  of  the  definition.   Twenty-two 
percent  were  identified  as  relying  on  a 
compensatory  accommodation,  48%  as  using 
or  needing  services  beyond  what  is 
expected  and  30%  were  identified  as 
having  functional  limitations.   Again, 
it  is  apparent  that  if  any  individual 
component  of  our  definition  had  been 
used  alone  to  identify  disabled 
children,  a  substantial  proportion  of 
children  would  have  been  mis-classified 
as  healthy. 

In  the  next  analysis,  we  examined 
whether  children  identified  by  different 
components  of  the  definition  would  have 
different  diagnoses  and  conditions. 
This  information  was  obtained  during 
field  testing  from  the  optional  query 
described  earlier.   Respondents  were 
asked  to  name  the  condition  that  caused 
the  particular  problem  for  the  child. 
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INNER-CITY  SAMPLE: 
PROPORTION  IDENTIFIED  AS  DISABLED 


2   COMPONENTS 


SERVICE   USE 


TOTAL  SAMPLE  IDENTIFIED  AS  DISABLED 
246/1275  (19%) 


Figure  2.   Proportion  Identified  as 

Disabled  in  Inner-City  Sample 


Note  that  we  were  not  always  able  to  get 
this  information.   Some  respondents  gave 
descriptions  rather  than  "diagnoses"  or 
said  they  simply  did  not  know  or  could 
not  remember  the  name  of  the  health 
condition.   This  underlines  one  of  the 
advantages  of  the  QuICCC:  children  can 
be  identified  independent  of  the 
respondent's  ability  to  name  the 
condition. 

Since  there  were  no  apparent 
differences  in  the  types  of  conditions 
children  had  in  the  National  and  Inner- 
City  Samples,  we  combined  the  two  data 
sets  for  this  analysis.   Children  who 
were  identified  only  by  functional 
limitations  had  conditions  such  as 
hearing  impairments  and  deafness,  visual 
impairments  and  blindness,  diabetes, 
asthma,  and  epilepsy.   Children 
identified  only  by  service  use  or  need 
beyond  that  expected  had  conditions  such 
as  behavioral  and  emotional  disorders, 
multiple  sclerosis,  sickle  cell  anemia, 
asthma  and  epilepsy.   Children 
identified  only  by  their  reliance  on 
compensatory  accommodations  had 
conditions  that  included  heart 
conditions,  high  cholesterol,  hyper- 
tension, sickle  cell  anemia,  asthma  and 
epilepsy. 

There  is  a  considerable  degree  of 
overlap  in  the  types  of  conditions  found 
for  children  identified  by  the  three 
components  of  our  noncategorical  defini- 
tion of  disability.   Children  with 
severe  conditions  are  included  in  each. 
For  example,  children  with  asthma  and 
epilepsy  were  reported  to  have  a  wide 
range  of  different  consequences  for  the 
same  condition.   Some  children  were 
restricted  in  their  activities  or  had 
difficulty  performing  tasks  of  daily 
living.   Others  were  hospitalized,  had 
frequent  medical  visits  or  needed 
nursing  care  or  special  school  services. 
And  still  other  children  with  the  same 


condition  were  able  to  compensate  and 
did  not  suffer  from  functional  limita- 
tions or  increased  use  or  need  of 
services,  but  did  reguire  medication, 
maintenance  of  special  diets  or 
assistive  devices. 

Children  identified  by  all  three 
components  of  our  definition  of 
disability  tended  to  have  more  pervasive 
conditions  such  as  Down  syndrome  and 
other  forms  of  mental  retardation, 
cerebral  palsy,  autism,  deafness,  spina 
bifida,  and  a  variety  of  birth  defects. 
These  children  also  tended  to  have 
multiple  conditions  such  as  heart  and 
kidney  conditions,  asthma  and  epilepsy. 

FUTURE  PLANS 

Future  analysis  plans  include 
further  exploration  of  the  character- 
istics of  children  identified  by 
different  components  of  our  noncate- 
gorical definition  of  disability.   We 
will  describe  demographic  character- 
istics including  age  distributions  and 
gender,  socioeconomic  status,  impact  on 
the  family,  federal  program  eligibility 
and  out-of-pocket  expenses.   Second, 
other  data  sets  will  become  available 
for  analysis  in  the  near  future.   For 
example,  the  QuICCC  is  currently  part  of 
a  statewide,  cross-sectional  household 
survey  of  children  in  the  state  of 
Arizona,  including  a  sample  from  the 
Native  American  population.   We  will 
repeat  the  same  analyses  in  this  data 
set.   Finally,  we  plan  to  combine  all 
three  data  sets  together  to  assess 
whether  there  are  racial  and  ethnic 
differences  in  the  prevalence  of 
disability  as  we  have  defined  it,  both 
overall,  and  by  each  of  the  three 
definitional  components. 

CONCLUSION 

This  paper  reports  preliminary 
results  from  an  ongoing  project  designed 
to  examine  the  policy  implications  of 
using  alternative  functional  definitions 
of  disability  to  identify  children  for 
research,  policy  or  programmatic 
purposes.   As  this  study  progresses,  we 
will  consult  with  child  health  policy 
experts  to  help  us  draw  out  the  policy 
implications  of  our  data.   At  this  time, 
we  have  two  concluding  remarks. 

First,  in  using  the  definition  of 
disability  that  we  developed,  we  found 
that  a  significant  proportion  of 
disabled  children  would  have  been 
misidentif ied  as  healthy  if  any  one 
component  of  our  definition  were 
omitted.   This  finding  can  only  be 
interpreted  with  respect  to  our  partic- 
ular operationalization  of  disability  as 
reflected  in  the  QuICCC.   But  it  does 
raise  a  larger  issue.   Obviously,  for 
appropriate  fiscal  planning,  we  do  not 
want  to  underestimate  the  number  of 
children  with  chronic  and  disabling 
conditions.    Who  will  be  classified  as 
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"disabled"  and  who  will  not  is 
completely  dependent  on  how  we  define 
disability.   For  different  purposes, 
different  definitional  approaches  and 
criteria  will  need  to  be  developed.   A 
comprehensive  definition  of  disability 
for  children  that  covers  all  concepts 
relevant  to  its  application  should  help 
avoid  some  of  the  pitfalls  inherent  in 
underestimating  prevalence. 

Second,  children  identified  with 
reliance  on  compensatory  mechanisms  or 
assistance  may  be  a  strategic  group  to 
examine  from  a  disability  policy  per- 
spective.  Recent  legislation  relating 
to  the  American's  with  Disabilities  Act 
of  1990  mandates  a  broader  perspective 
with  regard  to  the  disabled  community  by 
addressing  issues  relevant  to  people  who 
have  ongoing  health  conditions  with 
consequences,  including  those  not 
directly  experiencing  disability  in  the 


classic  sense, 


We  see  the  dimension  of 


reliance  on  compensatory  accommodation 
as  a  critical  component  to  our  defini- 
tion of  disability,  and  we  believe  it 
may  have  enormous  implications  for 
planning  and  funding  decisions  for 
children  with  disabilities. 
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YOUTH  WITH  RUNAWAY  EXPERIENCES 
IN  THE  HOUSEHOLD  POPULATION 


Chris  Ringwalt,  Research  Triangle  Institute 


The  purpose  of  my  talk  today  is  to  present 
prevalence  estimates  of  the  number  of  youth  in 
the  household  population  with  runaway 
experiences,  as  well  as  correlates  of  runaway 
experiences .   Runaway  adolescents  are 
particularly  vulnerable  for  such  negative 
physical,  psychological,  and  social  health 
outcomes  as  substance  use,  suicide,  sexual 
exploitation,  and  physical  violence.   Runaway 
youth  face  a  multitude  of  dangers  while  away  from 
home.   That  they  need  a  wide  variety  of  services 
is  unquestionable,  as  are  the  high  costs  they 
will  impose  on  society  if  their  needs  are  not  met 
effectively.   It  is  also  clear  that  their  needs 
are  not  now  being  met  adequately,  and  that  the 
limited  services  they  currently  receive  are  in 
danger  of  eroding  further.   Hence,  they  also  meet 
the  other  criterion  of  vulnerable  populations 
mentioned  in  Dr.  Richard  Brown's  presentation 
this  morning. 

Our  objectives  in  this  study  are  twofold: 

•      to  estimate  the  number  of  youth  in 

the  household  population  who  reported 
previous  runaway  experiences,  as  well 
as  the  number  of  runaway  youth  who 
may  have  also  had  homeless  episodes, 
and 

to  estimate  drug  use  and  other  at- 
risk  behaviors  among  youth  with 
previous  runaway  experiences  and 
compare  these  estimates  to  those  of 
youth  who  reported  no  runaway 
experiences . 

Only  one  cost-effective  methodology 
suggested  itself  as  a  means  to  meet  these 
objectives:   to  add  questions  concerning  runaway 
experiences  and  homeless  episodes  to  a  national 
household-based  survey  of  youth  that  included 
questions  about  drug  use  and  other  risky 
behaviors.   As  a  further  requirement,  the  survey 
needed  to  be  of  sufficient  size  that  a  reasonable 
number  of  youth  would  respond  positively  to 
questions  about  relatively  rare  runaway 
experiences  and  homeless  episodes.   As  we  were 
aware,  too  small  a  sample  would  prevent 
meaningful  comparisons  to  youth  without  these 
experiences . 

We  were  thus  fortunate  to  be  able  to  add 
questions  concerning  these  epxeriences  to  the 
Youth  Risk  Behavior  Supplement  (YRBS),  which  was 
conducted  as  a  follow-up  to  the  National  Health 
Interview  Survey  (NHIS) .   For  the  YRBS,  a  sample 
of  youth  aged  12  to  21  was  drawn  from  the  49,401 
families  interviewed  for  the  NHIS.   Within  each 
family  selected,  one  youth  attending • school  and 
up  to  two  youth  not  in  school  were  selected.   Of 
the  13,789  youth  sampled,  interviews  were 
completed  with  10,645  adolescents,  yielding  a 
final  response  rate  of  77.2%.   Of  these,  we 
analyzed  the  responses  of  all  6,497  youth  aged  12 
to  17,  choosing  as  the  upper  bound  the  age  beyond 
which  youth  are  no  longer   considered  runaway. 

Data  collection  for  the  YRBS  covered  a 
12-month  period  from  April  1992  through  March 
1993.   Surveys  were  conducted  by  means  of  taped 
interviews  delivered  by  audiocassettes .   Youth 
listened  to  the  interviews  and  recorded  their 
answers  on  forms  that  included  response  options 
but  not  the  questions  themselves.   Upon 
completion  of  the  interview,  youth  returned  the 
answer  sheets  in  sealed  envelopes  to 


interviewers.   This  procedure  assisted  youth  of 
marginal  literacy  and  protected  the  privacy  of 
all  adolescents'  answers  from  potentially  prying 
eyes  of  family  members. 

The  questions  we  added  to  the  YRBS  may  be 
found  in  Exhibit  1.   The  question  we  asked 
concerning  running  away  is  quite  conventional  and 
has  been  used  elsewhere  (e.g.,  Finkelhor, 
Hotaling,  &  Sedlak,  1990)  .   However,  there  is 
much  less  consensus  in  the  research  community 
concerning  how  to  operationalize  homelessness  for 
adolescent  populations.   We  asked  youth  about  key 
locations  where  youth  might  have  spent  the  night 
(or  the  "where"  of  homelessness).   However,  for 
reasons  of  space,  we  were  constrained  from 
tapping  the  temporal  dimension  of  the  experiences 
of  these  youth  (or  the  "how  long"  of 
homelessness) .   Therefore,  these  questions  should 
be  considered  indicators  of  homelessness. 

Exhibit  1 
Questions  Concerning  Runaway  Experiences  and  Homeless 
Episodes 

Runaway  Experience 

During  the  past  12  months,  have  you  stayed  out 
overnight  without  permission? 

Homeless  Episodes 

During  the  past  12  months,  did  you  spend  at  least 
one  night  in: 

1  a  youth  or  adult  shelter? 

2  a  public  place,  like  a  train  or  bus  station,  a 
restaurant,  or  an  office  building? 

3  an  abandoned  building? 

4  a  car,  truck,  or  van? 

5  a  park,  on  the  street,  under  a  bridge  or 
overhang,  or  on  a  roof  top? 

6  did  you  go  home  with  someone  you  did  not  know 
because  you  needed  a  place  to  stay? 

7 the  subway,  or  other  public  place  underground? 

We  now  turn  to  our  findings  as  they  relate 
to  the  objectives  we  set  forth  earlier.   We  have 
made  several  key  comparisons,  which  are  depicted 
in  Exhibit  2.   To  understand  this  exhibit,  it  is 
important  to  recognize  that  youth  could  have 
reported  a  runaway  experience,  or  any  one  of  the 
homeless  episodes  about  which  they  were  asked,  or 
both.   Hence,  the  two  circles  intersect  but  do 
not  overlap.   In  our  findings,  we  compare  youth 
who  indicated  they  had  run  away  to  those  who 
reported  neither  a  runaway  experience  nor  a 
homeless  episode.   We  also  compare  to  this  latter 
group  runaway  youth  who  reported  at  least  one 
homeless  episode  (the  gray  area  in  the  exhibit) 
and  runaway  youth  who  reported  no  homeless 
episodes  (the  black  area).   We  expected  that 
youth  with  both  runaway  experiences  and  homeless 
episodes  would  be  more  likely  to  demonstrate 
risky  behaviors  than  would  youth  with  runaway 
experiences  but  no  homeless  episodes.   Not 
included  in  our  comparisons  are  youth  who 
reported  at  least  one  homeless  episode,  but  no 
runaway  experiences.   (Note  that  we  have  no  way 
to  determine  whether  youth  who  reported  a 
homeless  episode  did  so  while  away  from  home 
without  permission.   That  is,  their  runaway 
experiences  and  homeless  episodes  could  have 
occurred  contemporaneously  or  at  different  times 
during  the  12-month  reference  period.) 
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Exhibit  2 
Definitions  of  Runaway  Experiences 


Runaway  Experience, 
No  Homeless  Episode 


Runaway  Experience, 
Any  Homeless  Episode 


In  Exhibit  3,  we  display  estimates  of  the 
number  of  youth  nationwide  in  the  household 
population  who  reported  runaway  experiences  in 
the  previous  12  months,  and  among  those  the 
number  who  also  reported  any  possible  homeless 
episodes  within  the  same  reference  period.   Note 
that  almost  one  out  of  seven  adolescents  reported 
at  least  one  runaway  experience,  and  that  of 
those  almost  one  out  of  five  also  reported  a 
homeless  episode.   Although  these  estimates  are 
higher  than  those  previously  reported  for  the 
adolescent  population  (Finkelhor  et  al .  ,  1990; 
Opinion  Research  Coporation,  1976),  they  are 
commensurate  in  magnitude  with  those  of  a 
recently  published  study  concerning  homelessness 
among  adults  (Link,  Susser,  Stueve,  Phelan, 
Moore,  &  Struening,  1994) . 

Exhibit  3 

Percentage  and  Estimated  Number  of  Youth 

Aged  12  to  17  Reporting  Runaway  Experiences 

(N=6,497) 


Runaway  Experiences 


Weighted 
11 


(SE) 


(SB) 


Runaway  experience, 
no  homeless  episode 


Runaway  experience, 
any  homeless  episode 


Any  runaway  experience 


2,270.849   (123,263)  12.0   (0.56) 

522,629     (49,894)   2.8    (0.24) 

2,793,478   (141,751)  14.8   (0.61) 


We  then  looked  further  at  youth  who 
reported  a  runaway  experience  and  the  location  of 
each  homeless  episode  in  the  previous  12  months 
and  estimated  the  number  of  each.   As  Exhibit  4 
indicates,  the  greatest  number  reported  that  they 
stayed  outside  (i.e.,  in  a  park,  on  the  street, 
under  a  bridge,  or  on  a  rooftop)  and  that  the 
least  number  reported  staying  in  a  subway  or 
underground  (questions  5  and  7  in  Exhibit  1, 
respectively) .   These  relative  responses  are 
about  what  we  expected,  given  the  ready 
availability  of  outside  locations  as  opposed  to 
the  relatively  small  number  of  urban  areas  with 
subway  systems .   These  responses  thus  suggest 
that  the  number  of  adolescents  who  answered 
questions  capriciously  was  quite  limited. 


Exhibit  4 

Estimated  Numer  of  Youth  Who  Reported  a 

Runaway  Experience  and  Location  of  Each 

Homeless  Episode 

(N«=6,497) 


Location* 

Unweighted 
N 

Weighted 

■ 

SE 

Outside 

61 

199,906 

(30,829) 

Public  place 

43 

146,583 

(24,734) 

Youth  or  adult  shelter 

41 

136,905 

(23,616) 

With  a  stranger 

37 

111,240 

(19,935) 

Abandoned  building 

34 

104,540 

(20,692) 

Subway 

6 

22,295 

110,316) 

We  now  turn  to  comparisons  of  risk 
behaviors  among  youth  with  runaway  experiences  to 
those  without  either  runaway  experiences  or 
homeless  episodes.   Ncte  in  Exhibit  5  that  that 
the  vertical  bars  represent  odds  ratios  for  past 
30-day  substance  use  that  have  been  adjusted  for 
adolescents'  age,  gender,  racial/ethnic 
background,  familial  poverty  status,  and  the 
population  density  of  the  area  in  which  they  live 
(metropolitan  statistical  area  [MSA]  status).   As 
you  might  expect,  the  tallest  bars  (i.e.,  the 
highest  odds  ratios)  are  youth  who  reported  both 
runaway  experiences  and  homeless  episodes.   The 
odds  ratios  associated  with  past  30-day  cocaine 
use  are  particularly  dramatic,  but  even  those 
indicating  elevated  risk  for  the  other  substances 
examined  (cigarettes,  alcohol,  and  marijuana)  are 
in  the  neighborhood  of  five  times  greater  for 
youth  with  runaway  experiences  than  for  those 
without.   The  odds  ratios  associated  with  other 
risk  behaviors  about  which  we  asked  (including 
carrying  guns,  knives,  and  clubs  in  the  past  3  0 
days,  physical  fighting  in  the  past  12  months, 
sexual  intercourse  in  the  past  3  months,  and 
lifetime  prevalence  of  sexually  transmitted 
diseases  [STDs] )  were  more  modest,  but  all 
exceeded  a  factor  of  3 .   Youth  aged  14  to  17 
reporting  both  runaway  experiences  and  homeless 
episodes  were  10  times  as  likely  to  report  a 
lifetime  STD  as  those  with  no  runaway 
experiences . 

Exhibit  5 

Past  30 -Day  Substance  Use  Among  Youth  Aged  12 

to  17  with  Runaway  Experiences 

(W=6,497) 


115 

110. 
35' 

30 

25 

20 

15 

10 

5 

0 


■■  Runaway  Experience 
Ea3  Runaway  Experience, 

No  Homeless  Episode 
^3  Runaway  Experience, 

Any  Homeless  Episode 


Cigarettes    Alcohol    Marijuana    Cocaine 


aAdjusted  for  age,  gender,  ethnicity,  metropolitan 
statistical  area  status,  and  poverty  and 
compared  to  no  runaway  experience . 

These  findings  serve  to  illustrate  the 
importance  of  administering  risk  behavior  surveys 
to  large,  nationally  representative  samples  of 
youth.   Such  samples  are  highly  effective  as  a 
means  to  tap  and  estimate  relatively  rare 
behaviors.   They  are  much  more  effective  than 
school  samples,  which  are  subject  to  bias  in  that 
they  do  not  include  those  who  are  often  the  most 
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likely  to  engage  in  the  behaviors  of  interest 
(i.e.,  dropouts).   Analyses  of  such  survey  data 
can  demonstrate  convincingly  the  increased  risk 
of  special  populations  of  interest  (e.g.,  runaway 
youth)  of  key  risk  behaviors  and  also  suggest 
specific  prevention  efforts  for  this  population 
(e.g.,  STD  and  cocaine  prevention).   Such  surveys 
can  also  serve  to  track  changes  in  the  prevalence 
of  special  populations  over  time  and  variations 
in  the  risk  behaviors  in  which  they  engage. 

There  is  simply  no  other  meaningful,  cost- 
effective  way  to  count  the  number  of  youth  with 
runaway  experiences  and  homeless  episodes  other 
than  by  a  survey  of  this  nature.   It  is  highly 
unlikely  that  a  nationally  representative  sample 
of  street  youth  will  ever  be  enumerated  because 
of  the  excessive  costs  involved.   It  is  also 
arguable  that  to  devote  scarce  public  resources 
to  counting,  as  opposed  to  serving,  runaway  and 
homeless  youth  would  be  unethical .   For  these 
reasons,  future  administrations  of  risk  behavior 
surveys  such  as  the  one  reported  here  deserve  our 
unequivocal  support. 
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IDENTIFICATION  AND  TREATMENT  OF  PEDIATRIC  ASTHMA  PATIENTS  ACROSS  SITES  OF  AN  INTEGRATED  SERVICE 

DELIVERY  AREA 

Jeff  Eickholt,  Mayo  Clinic,  Mayo  Foundation 
Charles  Darby,  Greg  Angstman,  Lynn  Kelley,  Steve  Hagedorn 


Introduction 

In  this  era  of  health  care  reform,  it 
has  become  very  important  to  deliver  the 
highest  quality  of  care  to  patients  and  at  the 
same  time  stabilize  or  reduce  the  costs 
associated  with  that  care.   In  this  paper,  we 
define  pediatric  asthma  and  describe  its 
various  medical,  social,  and  financial  effects 
on  children  and  their  families.   We  then 
outline  currently  recommended  treatment 
regimens  for  children  with  asthma.   Finally, 
we  will  describe  a  study  that  we  completed  to 
evaluate  the  patterns  of  asthma  medicine 
prescriptions  at  three  primary  care  sites 
within  our  institution.   The  goal  of  this 
study  was  to  support  the  implementation  of  a 
clinical  guideline  intended  to  improve  the 
quality  of  care  given  to  pediatric  asthma 
patients  at  our  institution.   In  improving  the 
care,  we  hope  to  improve  the  health  of 
children  with  asthma  and  also  lessen  the 
financial  burden  by  lowering  the  number  of 
emergency  room  visits  and  hospitalizations 
related  to  asthma. 

Asthma  is  a  lung  disease  that  is 
characterized  by  airway  obstruction  that  is 
reversible  in  most  patients,  either 
spontaneously  or  with  treatment.   Other 
•  characteristics  of  asthma  include  airway 
inflammation,  decreased  airway  capacity,  and 
increased  airway  responsiveness  to  a  number  of 
stimuli,  including  exercise,  cold  air,  smoke, 
dust,  and  other  allergens.   Clinical 
manifestations  associated  with  asthma  include 
coughing,  wheezing,  tightness  of  chest,  and 
difficulty  in  breathing.   Signs  and  symptoms 
vary  greatly  between  individuals,  depending  on 
the  severity  of  asthma. 

The  exact  mechanisms  and  causes  of 
asthma  are  still  not  completely  understood  and 
there  is  currently  no  known  cure  for  this 
disease.   Recent  research,  however,  has 
identified  medication  regimens  and  treatment 
protocols  that  allow  physicians  to  better 
manage  children  with  asthma.  A  1991  expert 
panel  report1  on  this  condition  outlined 
guidelines  for  the  management  of  asthmatic 
patients  using  these  medications  and  other 
methods . 

Affects  of  Asthma 

Asthma  is  the  leading  chronic  disease 
among  children4  affecting  4  to  6  %  of  children 
in  the  United  States.2'3   Prevalence  rates  are 
highest  among  males,  blacks,  and  children  of 
households  earning  less  than  a  poverty  level 
income.2   The  prevalence  rate,  for 
undetermined  reasons,  has  risen  30  to  40  % 
during  the  1980'  s.2'5   Despite  decades  of 
extensive  research  that  has  identified  methods 
of  better  managing  asthma  patients,  the 


average  asthma-related  mortality  rate  for 
children  aged  5-14  years  increased  10  percent 
per  year  from  1980  to  1987. 6 

The  quality  of  the  lives  of  children  can 
be  greatly  impacted  by  asthma.   It  has  been 
estimated  that  as  many  as  30%  of  children  with 
asthma  have  limited  activity  and  that  children 
with  asthma  miss  on  average  twice  as  many 
school  days  as  non-asthmatic  children.7 

Asthma  has  a  great  physical  and 
psychological  effect  on  children  and  also  a 
great  financial  impact  upon  their  families. 
Asthma  is  a  chronic  condition  that  requires 
regular  visits  to  physicians  and  regular  use 
of  medications.   Some  exacerbations  of  asthma 
may  require  emergency  room  visits  and 
hospitalizations.   It  has  been  estimated  that 
children  have  nearly  13  million  contacts  with 
health  care  providers  each  year  directly 
attributable  to  asthma.2   In  addition,  there 
are  150,000  to  200,000  asthma-related 
hospitalizations  each  year  among  children.2'4 

Severity 

The  severity  of  asthma  varies  greatly 
from  child  to  child  and  is  a  key  factor  when 
considering  therapy.   Mild  asthma  is 
characterized  by  exacerbations  that  are 
infrequent  and  usually  short  in  duration. 
These  exacerbations  usually  resolve 
spontaneously  or  with  medication  that  can  be 
administered  by  the  patient,  or  caretaker, 
without  a  physician  visit.   Between 
exacerbations,  mild  asthmatics  show  few  signs 
or  symptoms  and  tolerate  moderate  exercise 
well,  but  may  become  symptomatic  with  vigorous 
exercise. 

Children  with  moderate  asthma  have  more 
serious  and  frequent  exacerbations.   These 
episodes  may  occur  more  than  twice  per  week 
and  the  symptoms  may  last  for  several  days. 
Between  exacerbations,  these  children  may 
continue  to  have  some  symptoms,  including 
coughing  and  wheezing.   The  moderate  asthmatic 
also  generally  experiences  diminished  exercise 
tolerance. 

Severe  asthma  is  characterized  by  an 
almost  continuous  coughing  and  wheezing.   The 
severe  asthmatic  can  experience  frequent, 
sudden,  and  severe  exacerbations  and  will 
often  need  urgent  medical  care  to  bring 
exacerbations  under  control.   The  severe 
asthmatic  has  very  poor  exercise  tolerance. 

Recommended  treatment 

In  1991,  an  expert  panel1  published 
guidelines  for  the  diagnosis  and  management  of 
asthma.   This  report  specifically  addressed 
the  treatment  and  management  of  children  with 
asthma.   It  describes  the  following  goals  of 
therapy:  (1)  maintain  normal  activity  levels; 
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(2)  maintain  "normal"  pulmonary  function;  (3) 
prevent  chronic  and  troublesome  symptoms;  (4) 
prevent  recurrent  exacerbations  of  asthma;  (5) 
avoid  adverse  effects  of  asthma  medications. 

The  key  components  of  asthma  management 
include  education,  avoidance  of  triggers, 
monitoring,  and  pharmacologic  therapy.   For 
this  project  we  restricted  our  study  to 
pharmacological  asthma  management.   In  keeping 
with  the  panel  report,  which  details  various 
individual  treatment  regimens,  it  is  important 
to  reiterate  that  no  single  treatment  regimen 
can  or  should  be  prescribed  for  all  children 
with  asthma.   Regimens  are,  rather,  tailored 
to  the  individual  after  physician  examination 
and  evaluation,  and  treatment  is  based  on 
asthma  severity.   Because  severity  levels  may 
change  over  time,  it  is  important  that  the 
physician  evaluate  and  document  the  degree  of 
severity  regularly,  and  adjust  treatment 
accordingly. 

A  child  with  mild  asthma  should 
generally  receive  a  beta-2  agonist,  to  be  used 
as  needed  to  alleviate  acute  exacerbations . 
Beta-2  Agonists  act  by  reducing  airway 
obstruction. 

Moderate  asthmatics  should  receive  beta- 
2  agonists  to  be  used  on  a  regular  basis.   In 
addition,  they  should   receive  one  or  more 
anti- inflammatory  medications,  such  as 
cromolyn  sodium  or  a  corticosteroid.   These 
medications  should  be  used  on  a  regular  basis 
to  prevent  and  reduce  inflammation  of  the 
airways . 

A  child  having  severe  asthma  requires  a 
more  aggressive  approach.  It  is  recommended 
that  these  patients  receive  a  beta-2  agonist 
to  be  used  on  a  regular  basis.  In  addition, 
corticosteroids  and  cromolyn  sodium  are  also 
recommended.  This  class  of  patient  may  also 
need  additional  medications,  such  as 
theophylline,  to  control  their  asthma. 

Methodology 

The  Mayo  Clinic  is  made  up  of  a  number 
of  different  types  of  clinical  care  providers, 
including  primary  care  sites,  pediatrics, 
allergy  specialty  areas,  urgent  care  centers, 
and  affiliated  hospitals.   Children  with 
pediatric  asthma  can  be  seen  at  any  one  of 
these  sites  or  at  combinations  of  the  sites. 
In  this  study,  we  used  patient  billing  data  to 
identify  children  with  a  primary  diagnosis  of 


asthma  who  were  seen  in  at  least  one  of  three 
primary  care  sites.   These  sites  included 
family  medicine,  pediatrics,  and  our  pediatric 
allergy  center.   We  also  identified  patients 
seen  in  three  of  our  acute  care  areas.   These 
included  an  urgent  care  center,  emergency 
room,  and  a  hospital. 

Our  clinical  nurse  abstractors  then 
reviewed  the  histories  of  these  patients  and 
recorded  information  pertinent  to  each  asthma 
patient,  including  additional  visits, 
prescribed  medications,  severity 
documentation,  diagnostic  tests,  and  other 
clinical  detail  for  the   four-month  period 
from  September  1993  through  December  1993. 

Results 

During  these  four  months,  we  identified 
573  asthmatic  children  that  were  seen  in  at 
least  one  of  the  six  sites  mentioned  above. 
These  patients  averaged  1.9  visits  each  for  a 
total  of  1,119  asthma-related  visits  to  the 
six  sites  during  this  four  month  period.   The 
children  ranged  in  age  from  3  months  to  16 
years (mean=7 . 5  years,  S.D.=4.2)  and  61%  were 
males.   Asthma  severity  was  documented  in 
fewer  than  ten  percent  of  patient  medical 
histories  during  the  selected  period. 

We  categorized  the  patients  according  to 
which  of  the  three  primary  care  sites  they 
visited.   Table  1  summarizes  information  about 
the  patients  by  their  primary  care  site. 
Sixty-nine  percent  of  the  patients  were  being 
seen  exclusively  at  one  of  the  three  primary 
care  sites.   An  additional  6%  were  seen  in 
more  than  one  of  the  sites.   Twenty-five 
percent  of  the  patients  were  seen  only  in  an 
urgent  care,  emergency  room,  or  hospital 
setting  and  had  no  visits  to  the  three  primary 
care  sites  during  these  four  months. 

We  were  primarily  interested  in 
medication  prescription  patterns  at  the  three 
primary  care  sites,  so  we  restricted  our 
analysis  to  the  430  patients  with  one  or  more 
primary  care  visits.   For  each  of  these  sites, 
including  the  group  seen  at  multiple  sites,  we 
examined  all  primary  care  visits  for  each 
child  and  recorded  which  medications  had  been 
prescribed  at  any  time  during  the  study 
period.   Ninety-four  percent  of  the  patients 
had  some  type  of  asthma  medication  prescribed 
during  this  time  period.   Of  the  patients  with 
a  medication  prescribed,  87%  could  be  grouped 


Table  1:   Patient  Characteristics  by  Primary  Care  Site 


Site 

Numt 

er   of 

Average   Number 

Average 

Age 

Child 

ran   (%) 

of 

Vis 

its 

Family  Medicine 

41 

(7.2) 

2.4 

7.7 

Pediatrics 

118 

(20.6) 

2.5 

6.4 

Allergy 

237 

(41.4) 

1.9 

8.2 

Multiple  Primary  Care 

Sites 

34 

(5.9) 

5.3 

5.2 

No  Primary  Care* 

143 

(25.0) 

1.4 

7.8 

'Patients  seen  only  at  urgent  care  site,  emergency  room,  or  hospital. 
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into  five  standard  or  recommended  treatment 
regimens:  Beta-2  agonist  alone,  Beta-2  agonist 
+  oral  steroid,  Beta-2  agonist  +  inhaled 
steroid,  Beta-2  agonist  +  cromolyn  sodium,  and 
Beta-2  agonist  +  cromolyn  sodium  +  steroid  in 
Figure  1.   The  other  13%  included  patients  who 
did  not  receive  a  beta-2  agonist  (6%), 
patients  that  received  a  xanthine  (3%),  and 
patients  that  received  both  oral  and  inhaled 
corticosteroids  (4%) . 

Ninety-four  percent  of  treated  patients 
received  a  beta-2  agonist.   However,  the 
patients  seen  in  family  medicine  and 
pediatrics  received  the  beta-2  agonist  without 
other  medication  more  often  than  patients  seen 
in  the  allergy  site  or  in  multiple  sites.   The 
patients  seen  in  the  allergy  site  and  in 
multiple  sites  received  corticosteroids  and 
cromolyn  sodium  more  often  than  patients  seen 
in  the  other  two  sites. 

In  addition  to  medications,  we  also 
examined  the  number  of  emergency  room  visits 
and  hospitalizations  each  child  had  after  a 
primary  care  visit.   Fifteen  percent,  thirteen 
percent,  and  twenty-nine  percent  of  the  family 
medicine,  pediatrics,  and  the  combination 
group  patients,  respectively,  had  an  emergency 
room  visit  or  hospitalization,  while  only 
seven  percent  of  the  allergy  patients  visited 
the  emergency  room  or  hospital. 


Limitations 

There  are  limitations  to  our  results. 
First,  patients  were  only  followed  over  a  four 
month  period,  during  which  treatment  may  not 
have  represented  the  overall  treatment 
pattern.   Also,  we  did  not  separate  newly 
diagnosed  patients  from  those  with  chronic 
asthma  diagnoses,  which  may  have  skewed  the 
overall  picture  of  the  pattern  of  medications 
prescribed.   Finally,  since  asthma  severity 
information  was  rarely  documented,  we  were 
unable  to  correlate  level  of  severity  to 
medication  use,  or  to  directly  compare  asthma 
management  across  sites.   Despite  these 
limitations,  however,  our  study  revealed 
useful  and  interesting  information,  as 
described  below. 

Discussion 

National  guidelines  advise  that  all 
children  with  asthma  have  a  primary  care 
provider  to  manage  their  condition.   Twenty- 
five  percent  of  the  children  that  we  studied 
had  no  asthma-related  visits  to  the  three 
primary  care  sites  we  examined.   These 
patients  may  have  a  primary  care  provider 
outside  of  our  system  or  may  not  have  a 
primary  care  provider  at  all.   Another  6%  had 


Figure  1:   Proportion  of  patients  that  were  prescribed  each  asthma  medication  regimen  by 
primary  care  site 
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visits  to  multiple  sites,  suggesting  that  they 
have  no  main  primary  care  provider  to  manage 
their  asthma.   A  key  component  of  the 
guideline  implementation  will  be  identifying 
children  seen  in  acute  care  areas  or  in 
multiple  primary  care  sites  that  have  no 
primary  care  physician  to  manage  their  asthma. 
Attempts  will  then  be  made  to  refer  these 
patients  to  a  primary  care  provider. 

The  guideline  also  suggests  that  the 
severity  of  the  asthma  should  be  documented  on 
a  regular  basis  in  order  to  properly  manage 
the  condition.   There  was  very  little 
documentation  about  the  severity  of  asthma  in 
this  group  of  patients.   Methods  to  improve 
this  documentation  will  be  included  in  the 
implementation  of  the  guideline.   Clearly 
indicating  the  severity  of  asthma  should  guide 
clinicians '  decisions  on  which  asthma 
medications  to  prescribe. 

There  were  differences  in  the  use  of 
medications  across  the  sites.   We  know  that 
many  of  the  more  severe  asthma  cases  will  be 
followed  by  the  asthma  specialists  in  the 
pediatric  allergy  center.   This  would  explain 
why  a  higher  percentage  of  patients  at  this 
site  were  prescribed  anti-inflammatory  agents 
and  corticosteroids.   However,  a  greater 
percentage  of  children  being  seen  at  family 
medicine  and  pediatrics,  which  had  similar 
treatment  patterns,  had  visits  to  the 
emergency  room  or  a  hospital,  suggesting  that 
these  sites  are  also  seeing  moderate  and 
severe  cases  of  asthma. 

In  considering  the  group  of  34  patients 
who  were  seen  at  multiple  sites,  most  were 
first  seen  in  family  medicine  or  pediatrics 
and  were  eventually  seen  by  a  specialist  in 
the  allergy  center.   These  patients  had  the 
highest  number  of  visits  overall,  the  highest 
rate  of  treatment,  and  also  had  the  highest 
rate  of  emergency  room  visits  and 
hospitalizations.   This  suggests  that  many  of 
these  patients  had  severe  asthma  that  may  not 
have  been  in  control.   This  makes  it  difficult 
to  compare  them  to  patients  seen  in  only  one 
site.   In  any  case,  the  majority  of  these 
patients  were  referred  to  the  pediatric 
allergy  center,  which  is  precisely  the 
management  advised  by  both  national  and 
institutional  guidelines. 


guideline  at  these  sites  will  better  meet  the 
treatment  needs  of  children  with  asthma. 
After  implementation,  these  measures  will  be 
repeated  on  a  regular  basis  to  assess  the 
effect  the  guideline  has  had  on  the  quality  of 
care  given  to  asthmatic  children. 
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Conclusion 

We  have  evaluated  pediatric  asthma 
patient  site  visit  patterns  and  also  the  use 
of  asthma  medications  at  our  institution.   In 
doing  so,  we  identified  specific  components  of 
the  proper  management  of  asthma  that  need  to 
be  improved.   We  have  also  shown  that  the  use 
of  medications  is  not  the  same  at  every  site. 
However,  since  we  cannot  control  for  severity, 
we  cannot  evaluate  compliance  with  recommended 
guidelines  or  make  comparisons  between  sites. 

These  results  suggest  that  there  exists 
substantial  variation  between  the  three 
primary  care  sites  evaluated  with  respect  to 
the  nationally  recommended  guidelines  for  the 
management  of  children  with  asthma.   We 
believe  implementation  of  our  clinical 
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INTRODUCTION 

In  this  paper  we  analyzed  data  from  the 
1993  National  Health  Interview  Survey  (NHIS)  and 
showed  that  work  disability  affects  1  in  8 
adults  25-64  years  of  age  in  the  United  States. 
We  also  showed  that  in  comparison  to  people 
without  work  disability,  people  with  work 
disability  are  less  likely  to  be  in  the  labor 
force,  more  likely  to  be  poor,  as  likely  to  have 
health  insurance,  less  likely  to  have  private 
health  insurance,  more  likely  to  have  a  usual 
source  of  medical  care,  and  less  likely  to 
receive  needed  and  timely  care. 

Work  in  this  area  using  earlier  NHIS  data 
has  been  done  by  Mitchell  P.  LaPlante1  at  the 
Institute  for  Health  &  Aging,  University  of 
California,  San  Francisco,  CA,  and  Robert 
Griss2,3  at  the  Center  on  Disability  and  Health, 
Washington,  DC. 
BACKGROUND 

The  National  Health  Interview  Survey 
(NHIS)  is  an  annual,  nationally  representative 
sample  survey  of  the  household  population 
sponsored  by  the  National  Center  for  Health 
Statistics.   Personal  interviews  are  conducted 
in  sample  households  by  specially  trained 
interviewers  employed  by  the  Census  Bureau.   The 
1993  NHIS  sample  consisted  of  about  43,000 
households  containing  about  110,000  persons4. 
The  NHIS  basic  questionnaire  contains 
comprehensive  questions  on  health  status  and 
health  care  utilization.   In  addition  to  the 
yearly  NHIS  Core  data,  special  studies  are  often 
done.   In  1993,  special  studies  were  done  on 
access  to  health  care  and   health  insurance,  as 
well  as  other  topics.  Those  topics  were  repeated 
in  1994  and  1995,  years  which  also  included  a 
very  detailed  questionnaire  on  disability.   The 
latest  available  data  linking  disability,  health 
insurance,  and  access  to  care  are  the  1993  data 
used  in  this  analysis.   The  first  <->f  the  1994 
data  with  the  more  detailed  infor    ion  on 
disability  is  scheduled  to  be  av    ble  Spring 
of  1996. 

As  defined  by  the  World  Heal',  h 
Organization,  disability  refers  Lc  the 
consequences  of  bodily  impairments  which  limit 
the  functional  performance  or  activity  of  an 
individual2.   Disability  information  is 
available  yearly  from  the  NHIS  Core  set  of 
questions  that  are  asked  about  limitation  of 
activities-  including  work  limitations,  and 
their  associated  chronic  conditions  or 
impairments.   A  chronic  condition  is  defined  as 
a  present  condition  occurring  3  months  or  more 
before  the  reference  date  of  the  interview  or 
belonging  to  a  group  of  conditions  considered  to 
be  chronic.4  An  impairment  is  any  loss  or 
abnormality  of  psychological,  physiological,  or 
anatomical  structure  or  function  within  the 
body.2 

All  persons  18  years  of  age  and  older  in 
the  family  are  asked  questions  about  whether 
they  have  any  impairments  or  health  problems 
that  keep  them  from  NOW  working  at  a  job  or 
business,  or  limits  the  amount  or  kind  of  work 
they  can  do.   If  a  work  limitation  is  reoorted, 
AND  it  is  associated  with  a  chronic  health 
condition  or  impairment,  the  person  is 
considered  to  have  a  work  disability.  The 
categories  of  work  disability  that  we  used  are: 
Unable  to  work;  Limited  in  the  amount  or  kind  of 


work;  and  Not  limited  in  the  amount  or  kind  of 
work.   Included  in  the  last  group  of  Not  work 
limited  are  those  people  who  may  have 
limitations  that  are  not  work  limitations,  and 
also  people  where  it  is  unknown  whether  they 
have  work  limitations.   We  limited  our  analysis 
to  persons  25-64  years  of  age  to  ensure  the 
analysis  of  full-time  labor  force  candidates. 
Persons  younger  than  25  and  over  than  65, 
respectively,  are  often  not  in  the  labor  force 
because  of  school  or  retirement. 
RESULTS 

In  1S93,  according  to  Table  1  and  Figure 
1,  15.6  million  people  age  25-64  had  work 
disabilities;  they  made  up  12%  of  the  population 
in  that  age  group.   A  total  of  56%  of  the  "work 
disabled"  or  8.8  million  people  were  unable  to 
work  and  44%  or  6.8  million  were  limited  in  the 
amount  or  kind  of  work  they  could  do. 

People  with  work  disabilities  are  more 
likely  to  be  poor.   In  1993,  only  16%  of  people 
with  no  work  disability  had  incomes  less  than 
$20,000;  where  as  48%,  or  almost  half  of  those 
unable  to  work  had  incomes  less  than  $20,000. 
The  percent  of  people  who  are  limited  in  the 
amount  or  kind  of  work  with  incomes  less  than 
$20,000  is  29%,  between  the  two  other  groups. 
About  47%  of  people  with  no  "work  disabilities" 
have  incomes  of  $35,000  or  greater,  compared 
with  16%  of  people  unable  to  work  (Table  1) . 

One  reason  work  disability  is  associated 
with  low  income,  of  course,  is  that  persons  with 
work  disability  are  less  likely  to  work  or  to 
work  at  well-paying  jobs.   Whereas  84  percent  of 
persons  without  a  work  disability  are  in  the 
labor  f orce--either  employed  or  seeking 
employment--only  78  percent  of  persons  with  a 
limitation  in  the  kind  or  amount  of  work  are  in 
the  labor  force,  and  of  those  reported  to  be 
unable  to  work,  only  18  percent  are  in  the  labor 
force  (Table  1) . 

Because  most  persons  receive  health 
insurance  coverage  through  their  place  of 
employment,  it  is  not  surprising  that  work 
disability  is  associated  with  health  insurance 
coverage.   Persons  with  a  work  disability  are  as 
likely  to  have  health  insurance,  but  the 
important  difference  is  in  the  type  of  coverage: 
whereas  73  percent  of  persons  without  a 
disability  are  covered  by  a  private  insurance 
policy,  only  38  percent  of  persons  unable  to 
work  have  private  coverage;  on  the  other  hand, 
hardly  any  persons  without  a  disability  are 
covered  by  public  or  other  health  insurance 
coverage,  but  nearly  two-fifths  of  those  unable 
to  work  depend  on  these  two  categories  (Table 
1) .   Public  HI  coverage  includes  Medicaid  and 
other  public  assistance;  Other   HI  Coverage 
includes  Medicare,  Military  Health  care  and 
health  care  from  the  Indian  Health  service. 
Persons  unable  to  work  are  therefore  much  more 
likely  to  depend  on  these  public  programs  for 
their  medical  care. 

Health  insurance  coverage  can  affect 
access  to  health  care  through  its  affect  on  a 
person's  having  a  usual  source  of  sick  care.   A 
very  large  proportion  of  persons  with  health 
insurance  coverage  report  that  they  have  a  usual 
place  that  they  go  when  they  are  sick  or  need 
medical  advice,  and  this  is  true  regardless  of 
the  kind  of  insurance  they  have--private, 
public,  or  other--and  regardless  of  their  work 
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disability  status  (Figure  2).   Among  persons  who 
have  no  health  insurance  coverage,  a 
significantly  smaller  proportion  have  a  usual 
source  of  care. 

Among  those  with  no  health  insurance 
coverage,  however,  work  disability  is  associated 
with  an  increase  in  the  probability  of  having  a 
usual  source  of  care.   Among  persons  without  a 
work  disability,  58  percent  of  those  without 
coverage  had  no  usual  source  of  care;  but  among 
those  unable  to  work  and  no  coverage,  77  percent 
had  a  usual  source  of  care  (Figure  2).   This  may 
be  true  because  persons  who  are  work  disabled 
may  have  more  need  of  a  usual  source  of  care. 

We  turn  now  to  two  more  direct  measures  of 
access  to  care  among  persons  with  work 
disabilities:  not  getting  care  that  was  needed 
during  the  past  year,  and  delaying  medical  care 
because  of  worry  about  the  cost.   Because  health 
insurance  and  having  a  usual  source  of  sick  care 
affect  access  to  care,  we  considered  disability 
and  access  within  four  insurance  and  source 
categories:  has  health  insurance  and  a  usual 
source  of  sick  care  (HI  w/  USC) ;  has  insurance 
but  no  usual  source  of  care  (HI  w/o  USC) ;  has  no 
insurance  but  has  a  usual  source  (No  HI  w/  USC) ; 
and  has  neither  health  insurance  nor  a  usual 
source  of  care  (No  HI  w/o  USC) . 

Those  categories  in  that  order  show  a 
consistent  and  marked  relationship  to  not 
getting  needed  medical  care:  those  with  health 
insurance  coverage  and  a  usual  source  of  sick 
care  were  least  likely  not  to  get  needed  medical 
care,  and  those  with  neither  insurance  or  usual 
source  were  most  likely  not  to  get  needed  care 
(Figure  3).   The  relative  differences  are  very 
large:  those  in  the  least  favored  category  are 
6-12  times  more  likely  not  to  get  needed  medical 
care . 

Furthermore,  disability  is  related  to  not 
getting  needed  care  independently  of  insurance 
and  usual  source  of  care.  Whether  or  not  they 
had  insurance  or  a  usual  source  of  care,  persons 
with  a  work  disability  were  more  likely  not  to 
get  needed  medical  care  (Figure  3) . 

While  many  persons  with  work  disability 
reported  not  getting  needed  medical  care,  many 
more  reported  delaying  needed  medical  care  for 
financial  reasons.   While  both  health  insurance 
coverage  and  having  a  usual  source  of  medical 
care  are  related  to  financially  motivated  delays 
in  seeking  care,  health  insurance  is  the  more 
significant  factor  (Figure  4).   Again,  the 
effects  are  much  greater  among  those  with  a  work 
disability,  whether  moderate  or  severe:  more 
than  one-half  of  persons  with  a  work  disability 
and  no  health  care  coverage  reported  having 
delayed  getting  medical  care  for  financial 
reasons  during  the  past  year. 
DISCUSSION  AND  CONCLUSION 

In  conclusion,  we  showed  that  work 
disability  affects  1  in  8  adults  25-64  years  of 
age  in  the  United  States.  People  with  work 
disability  are  less  likely  to  be  in  the  labor 
force,  and  more  likely  to  be  poor. 

We  found  that  people  with  work 
disabilities  were  as  likely  to  have  health 
insurance  coverage  as  people  not  disabled  but  we 
found  their  type  of  coverage  was  different. 
People  with  work  disabilities  were  less  likely 
to  have  private  insurance  and  more  likely  to 
depend  on  public  programs  such  as  Medicaid, 
Public  Assistance  and  Medicare.   A  future  public 
policy  issue  may  be  extending  private  insurance 
coverage  to  more  people  with  disabilities.   That 
policy  issue  was  addressed  partially  in  1986 
when  legislation  was  enacted  which  made  it 
possible  for  people  to  continue  their  private 
health  insurance  coverage  after  leaving  a  job  by 
paying  both  the  employer  and  employee 
contributions  of  the  insurance  premiums2. 


Besides  health  insurance  coverage,  we 
looked  at  3  other  measures  of  access  to  care: 
having  a  usual  source  of  care;  needing  but  not 
getting  medical  care;  and  delaying  medical  care 
because  of  cost.   Although  they  are  important 
indicators,  these  measures  do  not  give  the  whole 
picture  of  adequate  health  care  access. 
Additional  information  on  quality  and 
appropriateness  of  care  are  needed. 

The  1993  NHIS  does  have  other  access  to 
health  care  information  that  we  have  not 
presented.   For  instance  there  are  several 
questions  asking  about  the  type  of  health 
professional  that  a  person  usually  sees  at  the 
usual  source  of  sick  care  and,  if  a  doctor, 
whether  or  not  the  doctor  is  a  specialist  or 
general  practitioner.   There  are  also  questions 
on  consumer  satisfaction  at  the  place  of  usual 
care,  such  as  waiting  time  for  an  appointment, 
waiting  time  to  see  the  doctor  when  they  had  an 
appointment,  satisfaction  about  getting  the  care 
they  thought  was  needed,  and  whether  the  doctor 
was  able  to  make  referrals  to  other  specialists. 

Our  analysis  to  date  is  preliminary  and 
needs  to  be  expanded.   Besides  this  information 
from  the  1993  NHIS,  the  Disability  Supplement  of 
1994  and  1995  will  have  much  more  detailed 
disability  information.   There  are  also  other 
government  surveys  that  ask  about  health  care. 
For  instance,  detailed  cost  of  medical  care  is 
available  from  NMES,  the  National  Medical 
Expenditure  Survey.   Starting  in  1996,  NMES  (or 
MEPS,  as  it  will  be  renamed)  will  be  drawn  from 
the  NHIS  sample  and  this  will  allow  the  linking 
of  both  the  NMES  and  NHIS  data  files  in  the 
future. 
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iiiWr  1.  Work  DliaUllly  by  1'hipioynicnt,  Income,  and  Health 
Insurance  for  MIBCW  25-64;cars  of  ugc:    lulled  States,  1992 
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Figure  3 

Not  Getting  Needed  Care  by  Health 
Insurance  and  USC 
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Figure  4 

Delayed  Care  Due  to  Cost  by  Health 
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SETTING  UP  A  SUSTAINABLE  MODEL  SYSTEM  FOR  DISABILITIES 
SURVEILLANCE:   RHODE  ISLAND,  A  CASE  STUDY 


Joann  M.  Lindenmayer,  Brown  University  School  of  Medicine 

John  P.  Fulton 
David  Hamel 


INTRODUCTION 

The  context  in  which  we  considered 
disabilities  and  its  surveillance  is 
changing.  In  the  past  two  decades  many 
common  diseases  and  conditions  have  been 
reexamined  and  redefined.  Among  these 
are  chronic  diseases,  injuries,  and 
disabilities.  In  1963,  Webster's 
International  Dictionary  defined 
disability  as  "deprivation  or  lack, 
especially  of  physical,  intellectual,  or 
emotional  capacity  or  fitness,  and  the 
inability  to  pursue  an  occupation  or 
perform  services  for  wages  because  of 
physical  or  mental  impairment."  In  1976, 
Stedman's  medical  dictionary  defined 
disability  as  a  "medicolegal  term 
signifying  loss  of  function  and  earning 
power."  In  1991  the  Institute  of 
Medicine  (IOM)  defined  disability  as  a 
"limitation  in  physical  or  mental 
function,  caused  by  one  or  more  health 
conditions,  in  carrying  out  socially 
defined  tasks  and  roles  that  individuals 
generally  are  expected  to  be  able  to  do." 
The  first  definition  considers 
disabilities  a  defect  in  the  individual, 
the  second  focuses  on  individual  loss  of 
function  and  earning  power,  and  the  last 
and  most  current  definition  acknowledges 
the  role  of  health  and  the  social 
environment  in  the  establishment  and 
maintenance  of  a  disability. 

Political  forces  have  shaped  and 
continue  to  shape  disabilities  prevention 
programs.  Federally-funded  disabilities 
prevention  programs  were  created  in  the 
late  1980s  under  a  categorical  funding 
scheme.  Recent  changes  in  the  political 
environment  will  usher  in  an  era  of  block 
grants.  Programs  that  address 
noninfectious  diseases  and  conditions 
will  not  only  be  forced  to  compete  for 
funding,  but  the  total  amount  of  funding 
will  be  considerably  less  than  it  has 
been. 

The  changing  definition  of 
disabilities  and  the  changing  political 
climate  will  have  wide-ranging  effects  on 
state  disabilities  programs  and  on 
surveillance  for  disabilities.  We  will 
use  the  experience  of  the  Disabilities 
Prevention  Program  in  Rhode  Island  to 
illustrate  these  points. 


AN    OVERVIEW    OF    THE    RHODE 
DISABILITIES  PREVENTION  PROGRAM 


ISLAND 


Rhode  Island's  Disability  Prevention 
Program  (RIDPP)  was  established  in  1991 
by  funding  from  the  Centers  for  Disease 
Control  and  Prevention  (CDC) .  The 
program's  original  focus  was  twofold  - 
the  primary  prevention  of  injuries  to 
children  and  facilitating  linkage  of 
children  to  services. 

Because  the  RIDPP  is  young  it  has 
only  recently  concluded  the  process  of 
developing  a  strategic  plan  for 
preventing  disabilities  in  Rhode  Island. 
The  focus  of  the  strategic  plan  is 
"preventing  disabilities  by  managing 
impairments."  A  plan  for  disabilities 
surveillance  is  included  in  the  strategic 
plan,  and  this  paper  constitutes  a  first 
draft  of  that  surveillance  plan. 


SETTING  UP  A  DISABILITIES  SURVEILLANCE 
SYSTEM  IN  RHODE  ISLAND 

Rhode  Island  has  not  been  invested 
in  the  disabilities  surveillance  process 
long  enough  to  be  committed  to  a 
particular  surveillance  system. 


General 
system 


criteria   for 


surveillance 


In  building  a  disabilities 
surveillance  system  we  first  considered 
four  general  criteria  recommended  for 
evaluating  an  existing  surveillance 
system1,  namely 

*  the  public  health  importance  of  the 
event, 

*  a  description  of  the  system, 
including  its  objectives,  definition 
of  events  under  surveillance,  and 
its  components  and  operation, 

*  the  usefulness  of  the  system  and 
what  actions  might  be  taken  as  a 
result  of  the  data,  and 

*  .   an  evaluation  of  the  system  for 

simplicity,  flexibility, 
acceptability,  sensitivity, 
predictive  value  positive, 
representativeness,  timeliness,  and 
the  resources  needed  to  operate  it. 
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Specific  surveillance  issues  related  to 
disabilities 

In  addition  to  these  general 
criteria  we  also  considered  the  unique 
nature  of  disabilities  surveillance. 

It  is  necessary  to  adapt  the  medical 
model  of  surveillance  for  noninfectious 
diseases  and  conditions.  Most 
surveillance  systems  have  traditionally 
focused  on  counting  occurrences  of 
readily  definable  vital  events  and 
infectious  diseases.  This  model  of 
surveillance  does  not  always  meet  the 
needs  of  noninfectious  diseases  and 
conditions.  To  illustrate  one  of  the 
many  problems  associated  with  applying 
traditional  surveillance  system  criteria 
to  disabilities  surveillance,  consider 
that  no  easily  standardized  clinical  or 
laboratory  criteria  can  be  used  to 
establish  the  case  definition  of  any 
particular  disability.  It  has  been  said 
that  "Although  little  disagreement  exists 
over  the  concept  of  impairment,  this  is 
not  true  for  the  concept  of  disability, 
as  persons  who  consider  themselves  to 
have  a  disability,  professionals  who 
study  disability,  and  the  general  public 
disagree  about  its  meaning.  It  is  no 
surprise  that  if  disability  is  perceived 
to  mean  different  things  to  different 
people,  demographic  counts  and  estimates 
of  the  number  of  persons  with  a 
disability  will  also  differ.   2 

It  will  be  necessary  to  integrate 
surveillance  for  disabilities  with 
surveillance  for  other  programs  such  as 
chronic  disease,  injury  control,  and 
maternal  and  child  health.  Because 
disabilities  prevention  cuts  across  so 
many  different  programs,  it  has 
historically  been  integrated  into  other 
categorical  programs.  Under  block  grants 
there  will  be  even  more  incentive  to 
strengthen  this  integration  in  order  to 
make  data  collection  as  efficient  as 
possible. 

Lastly,  some  measure  of 
accountability  must  be  built  into 
disabilities  surveillance  systems.  This 
process  has  already  been  applied  to 
prevention  block  grants,  which  will  make 
use  of  a  uniform  data  set  for  this 
purpose.   Disabilities  prevention  staff 

will  need  to  make  hard  decisions  about 
which  features  of  disabilities  to 
monitor.  The  Institute  of  Medicine  model 
of  disabilities  suggests  priorities  for 
disabilities  surveillance.  These  include 
enumeration  of  persons  at  points  along 
the  disability  continuum  and  assessing 
the  quality  of  their  lives  as  measures  of 
the  burden  of  disabilities  and  as 
indicators  of  the  success  of  disabilities 
prevention  and  intervention  programs. 


Assessing  the  current  state  of 
understanding  of  disabilities  in  Rhode 
Island 

As  the  first  step  towards  setting  up 
a  disabilities  surveillance  system,  we 
undertook  to  answer  the  question,  "How 
important  a  public  health  problem  is 
disability  in  Rhode  Island?"  by  reviewing 
what  was  known  about  the  extent  of 
disabilities. 

From  the  1990  Census,  work 
limitation/disability  among  16-64  year 
olds  was  7%  overall,  and  twice  this 
prevalence  was  found  in  the  cities  of 
Providence,  Pawtucket,  and  Woonsocket. 
Mobility/self  care  limitation  among 
persons  65  years  of  age  or  older  was 
18.3%  overall,  and  greater  than  25% 
prevalence  was  found  in  the  cities  of 
Providence,  Pawtucket,  Central  Falls,  and 
North  Smithfield. 

Every  five  years,  Rhode  Island 
conducts  the  Rhode  Island  Health 
Interview  Survey  (RIHIS) ,  a  household 
survey  modeled  after  the  National  Health 
Interview  Survey.  The  1985  RIHIS 
contained  questions  regarding  activity 
limitation  and  established  a  baseline 
against  which  to  measure  changes  in 
limitations.  From  the  1985  RIHIS  it  was 
determined  that  3.8%  of  children  under  5 
years  of  age  had  any  limitation  of  play 
activity,  10.4%  of  5-17  year  olds  had  any 
limitation  of  school  attendance,  14.1%  of 
18-64  year  olds  had  any  limitation  of 
work  or  other  activities,  and  33.2%  of 
persons  65  years  and  older  had  any 
limitation  of  their  capacity  for 
independent  living.  In  all  age  groups, 
the  proportion  with  the  most  extensive 
disability  ("unable  to  perform  a  major 
activity")  was  higher  in  Rhode  Island 
than  it  was  nationally. 


Two  years  ago  the  RIDPP  developed  a 
Disability  Databook3  using  existing  data 
sets.  Twenty  six  statewide  data  sets  were 
reviewed  for  the  following  criteria: 
*    reliability, 

documentation, 

computerized, 

population-based , 

representative , 

collects   age,   gender,   race   and 

census  tract, 

available  annually  from  1986  through 

1990,  and 

sufficient    cases    to    calculate 

meaningful  rates. 


Two  datasets  -  the  Hospital  Discharge 
Dataset  (HDDS)  and  the  1990  Census  -  were 
chosen  as  the  basis  for  the  databook. 
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The  authors  next  reviewed  four 
national  surveys  that  measured  functional 
limitations,  listed  the  top  10  disabling 
conditions  for  each  survey,  and  created  a 
comprehensive  list  of  21  disabling 
conditions.  From  these,  13  conditions 
associated  with  hospitalization  were 
chosen  for  inclusion  in  the  databook: 

*  five  chronic  diseases,  including 
cerebrovascular  disease,  diabetes, 
arthritis,  asthma,  and  colorectal 
cancer, 

*  three  injuries,  including  traumatic 
brain/spinal  cord  injury  (TB/SCI) , 
hip  fractures,  and  disc  injury,  and 

*  five  indicators  of  developmental 
disabilities,  including  congenital 
anomalies,  intrauterine  growth 
retardation,  prematurity,  birth 
hypoxia/asphyxia,  and  newborn 
hospital  length  of  stay  longer  than 
7  days. 

For  every  1000  births,  prematurity 
was  the  most  prevalent  discharge 
(70.3/1000)  among  those  with  some 
indication  of  developmental  disability, 
followed  by  prolonged  length  of  hospital 
stay  (36.6/1000),  hypoxia/asphyxia 
(19.7/1000),  and  intrauterine  growth 
retardation  (12.5/1000). 

Among  children  less  than  5  years  of 
age,  asthma  accounted  for  43.0 
discharges/10,000  children  and  congenital 
anomalies  for  38.5  discharges/10,000 
children. 


Pawtucket 


Providence 


Areas  with  highest  rates  and  burden 
of  disability-related  hospital  discharges 
Rhode  Island,  1986-1990 


Figure  1 . 


Among  persons  less  than  19  years  of 
age,  traumatic  brain  and/or  spinal  cord 
injury  accounted  for  9.1 
discharges/10,000  children. 

Among  adults  16-64  years  of  age, 
disc  injury  accounted  for  22.1 
discharges/10,000  adults. 

Among  adults  65  years  of  age  or 
over,  cerebrovascular  disease  accounted 
for  the  greatest  number  of  hospital 
discharges  (154.1  discharges/10,000 
adults),  followed  by  arthritis  (44.8 
discharges/10,000),  diabetes  (44.7 
discharges/10,000) ,  and  colorectal  cancer 
(37.6  discharges/10,000). 

Lastly,  among  white  women  65  years 
of  age  or  over,  hip  fractures  accounted 
for  93.4  discharges/10,000  women. 

Census  tracts  of  residence  were 
combined  into  14  geographic  groupings, 
ranked  by  rate  of  hospital  discharge  and 
by  the  total  number  of  people  affected. 
The  areas  that  ranked  in  the  top  half  of 
all  areas  for  both  rate  and  burden  are 
pictured  in  Figure  1.  They  include 
Providence  and  Pawtucket,  areas 
previously  described  as  having  the 
highest  rates  of  work,  mobility,  and 
self-care  limitation. 


Having  completed  preliminary 
analyses  with  existing  datasets,  we 
turned  our  attention  to  the  choice  of  a 
model  for  disabilities. 

Choice  of  a  Model  for  Disabilities 

For  purposes  of  disabilities 
surveillance  we  chose  to  adopt  the  IOM 
model  of  disabilities  with  the  following 
minor  modifications  (Figure  2) : 

*  We  added  stages  of  wellness  and 
death; 

*  We  added  arrows 
progress  along  the  continuum  from 
wellness  to  disability  is  reversible 
to  some  degree; 

*  We  regarded  the  attainment  of  each 
stage  as  a  risk  factor  for 
progression  to  the  next  stage. 
Hence  progression  to  any  stage 
implies  failure  of  risk  management 
at  the  last  stage.  This  is 
consistent  with  the  RIDPP  strategic 
plan. 


to  indicate  that 
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Model  for  Disabilities  Surveillance 
in  Rhode  Island 

(Adapted  from  the  Institute  of  Medicine,  1991) 
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Figure  2. 


Specification  of  the  Surveillance  System 

We  chose  the  "pathologic 
event/ impairment  stage"  as  the  starting 
point  for  our  surveillance  system  to 
limit  its  scope. 

Keeping  in  mind  that  our  strategic 
plan  calls  for  case  management  (a 
combination  of  medical  management  and 
linkage  to  social  services)  as  a  means  of 
preventing  disabilities,  we  developed  a 
matrix  defined  by  age  group  and  stage  in 
the  disabling  process  (Figure  3) .  It  is 
possible  to  categorize  current  sources  of 
disabilities  surveillance  data  using 
these  two  criteria.  For  example,  one 
surveillance  source  represented  on  the 
matrix  is  the  TB/SCI  registry. 
Currently,  the  registry  identifies 
persons  at  the  time  of  injury  (pathologic 
stage) ,  but  provides  no  followup  of 
potential  progression  along  the 
disability  continuum.  People  between  the 
ages  of  6  and  64  are  those  most  likely  to 
be  registered. 


Other  current  sources  of 
disabilities  surveillance  data 
represented  in  the  matrix  include  the 
comprehensive  prenatal  screening  program 
(CPSP) ,  newborn  screening  program  (NS) , 
early  intervention  program  (EI) ,  special 
education  programs  (SE) ,  hospital 
discharge  data  set  (HDDS) ,  Rhode  Island 
Health  Interview  Survey  (RIHIS) ,  and  the 
Behavioral  Risk  Factor  Surveillance 
System  (BRFSS) .  The  first  four  (CPSP, 
NS,  EI,  and  SE)  are  linked  together  and 
to  other  datasets  by  the  Rhode  Island 
Children's  Access  Program  (RICAP) ,  a 
system  that  integrates  public  health 
responsibilities  for  universal  newborn 
screening,  outreach,  home  visiting,  and 
follow-up  through  a  central  registry  and 
automatic  tracking  system. 

We  used  this  matrix  to  assess  the 
comprehensiveness  and  usefulness  of  our 
current  surveillance  sources.  Several 
observations  may  be  made: 

*  There  are  a  number  of  diverse 
sources  for  disabilities 
surveillance,  but  most  are  not 
specific  for  disabilities. 

*  Although  each  age  grouping  is 
covered  by  at  least  one  surveillance 
source,  the  disability  continuum  is 
best  represented  among  children. 

*  Few  linkages  are  found  among 
datasets. 

*  Many  datasets  are  limited  to 
counting  the  occurrence  of  an  event 
without  follow-up. 

*  None  of  the  sources  address  the 
needs  of  persons  without  telephones, 
most  of  whom  have  low  incomes  and 
are  therefore  at  high  risk  of  having 
or  developing  disabilities. 

We  used  the  matrix  to  identify  a 
number  of  steps  to  improve  the 
surveillance  system: 

*  Better  use  can  be  made  of  some 
sources  (e.g.,  TB/SCI)  to  follow 
persons  prospectively  through  the 
disability  continuum. 

*  Linkage  of  datasets  should  be 
considered,  particularly  among 
impaired  adults,  so  that  DPP 
services  can  be  targeted  and  used 
more  efficiently. 

*  Some  sources  (RIHIS,  BRFSS)  can  be 
adapted  to  assess  quality  of  life. 

*  Surveys  should  be  developed  to 
address  the  needs  of  those  who  are 
not  under  surveillance. 
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Disabilities  Surveillance  Matrix 
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BRFSS:    Behavioral  Risk  Factor  Surveillance  System;       TB/SCI:    Traumatic  Brain/Spinal  Cord  Injury  Registry 

Figure  3 


CONCLUSION 

This  paper  represents  Rhode  Island's 
first  steps  towards  setting  up  a 
sustainable  disabilities  surveillance 
system.  In  order  to  accomplish  this  we 
must  continue  to  challenge  established 
notions  about  surveillance  and  to  be 
responsive  to  the  changing  social  and 
political  environment. 
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A  RECORDS  -  SURVEY  COMPARISON  OF  ELIGIBILITY  AND  HEALTH  CARE 
UTILIZATION  MEASURES  FOR  MEDICAID  BENEFICIARIES:   ADULT  AND  CHILD  REPORTS 

Donna  Eisenhower  and  Angela  Schmitt,  Mathematica  Policy  Research,  Inc. 


Background 

This  study  is  part  of  a  larger  effort 
to  assess  the  feasibility  of  conducting 
a  national  survey  of  Medicaid 
beneficiaries.   The  study  has  been 
conducted  by  Georgetown  University's 
Center  for  Health  Policy  Studies  and 
Mathematica  Policy  Research  under  a 
grant  from  the  Physician  Payment  Review 
Commission.   A  national  survey  is  being 
considered  to  produce  state-specific 
estimates  and  generate  enough 
information  for  monitoring  access-to- 
care.   Many  sampling  and  other  survey 
design  issues  need  to  be  evaluated  to 
determine  the  feasibility  of  conducting 
such  a  survey.1  This  paper  reports  on  an 
effort  to  assess  the  accuracy  of  survey 
estimates  related  to  Medicaid 
eligibility  and  health  care  utilization 
for  adults  reporting  for  themselves  and 
for  adults  reporting  for  one  randomly 
selected  child.   The  accuracy  of  the 
survey  estimates  will  be  assessed  by 
comparing  the  survey  data  to  Medicaid 
records  data. 

The  survey  data  are  the  result  of  a 
pilot  survey  of  Medicaid  beneficiaries 
that  was  conducted  in  one  northeastern 
state  between  August  1  and  October  25, 
1993.   The  survey  resulted  in  358 
completed  adult  self-reports  and  320 
adult  reports  for  one  randomly  chosen 
child.   Medicaid  records  data  were 
abstracted  for  these  same  individuals 
after  waiting  an  additional  three  months 
to  assure  that  any  lagging  medical 
claims  were  processed. 

To  conduct  a  national  survey  requires 
that  eligibility  information  be 
accurately  reported  by  survey 
respondents  or  collected  from  records. 
If  the  accuracy  of  the  Medicaid 
eligibility  determination  is  in 
question,  all  measures  related  to  access 
or  utilization  will  be  meaningless  if 
such  measures  are  evaluated  in  terms  of 
Medicaid  participation.   This  matter  is 
complicated  in  Medicaid  populations 
where  a  portion  of  the  participants  have 
inactive  or  active  eligibility  (go  on  or 
off  Medicaid)  during  the  course  of  a 
year.   Therefore,  accessing  the  ability 
of  respondents  to  report  eligibility 
accurately  is  critical  to  evaluating  the 
feasibility  of  using  survey  data  or  the 
related  need  of  abstracting  such  data 
from  records. 

Secondly,  the  assessment  of  the 
accuracy  of  utilization  measures 
provides  a  measure  of  accuracy  in 
general  and  also  provides  a  basis  of 
assessing  the  feasibility  of  collecting 
accurate  measures  from  a  survey.   Recall 


bias  in  collection  of  health  care 
utilization  measures  is  well  documented 
in  the  literature.2  The  record- survey 
comparison  will  shed  light  on  the  need 
for  the  use  of  bounding  or  memory  aids,3 
the  appropriate  recall  period,  the  need 
for  statistical  adjustments  of  the 
estimates,  or  the  need  to  use  records 
data  for  certain  measures. 

The  measures  for  which  survey 
responses  will  be  compared  to  records 
data  for  both  the  self -report  of  the 
adult  and  the  adult  report  for  the  child 
include : 

•  Eligibility  for  Medicaid  by 
month  over  the  past  year 

•  Number  of  doctor's  visits  for 
either  last  month  or  last  three 
months4 

•  Date  of  admission  for  overnight 
hospitals  stays 

•  Number  of  nights  hospitalized 
Results 

Eligibility.   Several  measures  of 
eligibility  were  assessed  as  part  of  the 
full  study  and  are  available  as  part  of 
the  final  report.   The  most  important 
eligibility  question  asked  of  survey 
respondents  regarding  eligibility  was, 
"Now  thinking  back  over  the  last  twelve 
months  beginning  in  July  1993  and  ending 
in  August  1992,  please  tell  me  to  the 
best  of  your  ability  if  your  (then  the 
child's)  eligibility  for  Medicaid  was 
active  for  every  month  during  that 
year?"   As  Table  1  indicates,  91.8 
percent  of  survey  responses  for  adults 
matched  the  records  data.   Of  these, 
71.5  percent  were  active  for  all  twelve 
months  and  matched  and  20.3  percent 
matched  but  were  not  active  for  all 
twelve  months.   Of  the  8.2  percent  of 
responses  which  were  mismatches,  4.8 
percent  mismatching  was  because  the 
survey  said  they  were  active  when  the 
record  data  showed  they  were  not,  and 
3.4  percent  mismatching  was  because  the 
record  data  said  they  were  active  and 
the  survey  said  they  were  not.   At  89.0 
percent,  the  number  of  survey  responses 
of  the  adult  reports  for  the  child  was 
very  similar  to  those  for  adult  self- 
reports . 

While  this  is  a  modest  degree  of 
mismatch,  it  should  be  improved  if 
survey  data  alone  is  to  be  used  to 
produce  national  estimates.   Therefore, 
two  subsequent  analyses  were  conducted 
to  examine  the  reasons  for  mismatch. 
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The  first  examined  if  recall  bias 
explained  the  mismatches.   If  recall 
bias  alone  accounted  for  the  mismatches, 
more  mismatches  should  occur  further 
back  in  time.   While  there  was  a  pattern 
in  that  direction,  it  was  not  consistent 
for  either  the  adult  or  child  report. 

The  analyses  explored  the  concept  of 
"transition  month"  as  a  reason  for  the 
mismatch.   The  transition  month  is  the 
last  month  either  with  or  without 
Medicaid  and  the  ensuing  month  where 
there  is  a  change  of  status.   An 
analysis  of  this  phenomenon  showed  a 
mismatch  rate  for  adults  of  40.3  percent 
in  transition  months  compared  to  a  10.3 
percent  mismatch  rate  in  non- transition 
months.5  Similarly,  the  mismatch  rate  for 
the  adult  report  for  a  child  was  3  3.3 
percent  in  transition  months  compared  to 
10  percent  in  non- transition  months. 
Clearly,  the  greater  percentage  of 
mismatches  occur  during  a  transition 
month  when  a  Medicaid  beneficiary  may 
not  even  be  aware  that  their  official 
status  has  changed.   Regarding 
eligibility  status,  lack  of  awareness 
rather  than  recall  bias  is  more  of  an 
explanation.   This  actually  suggests 
that  the  survey  data  may  be  adequate  if 
the  data  users  account  for  the 
transition  month  phenomenon  when  looking 
at  other  survey  measures.   However,  the 
better  alternative  is  to  use  eligibility 
data  from  records  to  supplement  the 
other  survey  data. 

Utilization 

Doctor  Visits.   The  first  utilization 
question  to  be  compared  to  records  data 
read,  "During  the  last  month  (or  past 
three  months) ,  how  many  times  did  you 
see  a  medical  doctor  for  an  in-person 
visit?"6  As  mentioned,  one-half  the 
sample  responded  to  a  one-month  recall 
period  and  one-half  to  a  three-month 
recall  period.   The  findings  presented 
in  Table  2  for  adults  indicate  that  for 
exact  matches,  the  one -month  recall  is 
significantly  more  accurate  with  51 
percent  of  the  survey  responses  matching 
records  data  compared  to  26.9  percent 
for  the  three  month- recall  period. 

When  a  match  is  defined  reasonably  as 
±1  visit,  78.4  percent  match  for  the 
one-month  recall  for  adults  while  58.2 
percent  match  for  the  three -month.   Of 
adults  reporting,  83  percent  reported  0, 
1,  or  2  visits  within  the  last  month  (35 
percent  reported  1  visit) .   Within  the 
last  three  months,  of  adults  reporting 
75  percent  reported  0,  1,  or  2  visits 
(42  percent  reported  1  visit) .7 

Clearly,  a  one-month  recall  period 
for  doctor  visits  for  adults  is 
preferred  to  a  three-month  recall 
period.   In  fact,  if  accurate  estimates 
greater  than  ±1  visits  are  required,  a 
two-week  recall  period  may  be  necessary.8 


This,  however  would  suggest  that  a 
larger  sample  size  would  be  required. 
As  Table  2  also  indicates,  when  adult 
respondents  err,  more  (80  percent) 
overestimate  the  number  of  visits  at  a 
one-month  recall  period  while  slightly 
more  (55  percent)  underestimate  at  a 
three-month  recall  period.   This  is 
evidence  of  some  telescoping  at  a  one- 
month  period  and  some  recall  decay  at 
three  months.  This  also  suggests  that 
accuracy  would  be  improved  by  the  use  of 
bounding  techniques  for  either  a  one- 
month  or  two-week  recall  period. 

As  for  most  measures,  there  is  at 
least  slightly  greater  accuracy  for 
adults  reporting  for  a  child  than  for 
adults  reporting  for  themselves.   As 
Table  2  indicates,  exact  matches  for 
children  were  57.6  percent  for  the  one- 
month  recall  period  and  50.6  percent  for 
the  three -month  recall  period.   The  most 
noticeable  improvement  for  children  is 
at  the  three-month  period  with  50.6 
percent  compared  to  26.9  percent  for 
adults.   However,  even  50.6  percent  is 
less  than  the  desired  accuracy  for 
survey  estimates.   When  a  match  is 
defined  as  ±1  visit  for  children,  the 
matches  for  children  increase 
dramatically  to  93.2  percent  for  the 
one-month  recall  period  and  to  78.5 
percent  for  the  three-month. 

If  a  match  is  defined  as  ±1  visit  for 
children,  the  one-month  recall  period 
provides  a  good  survey  estimate  of  the 
number  of  doctor  visits.   An  exact  match 
could  probably  be  improved  further  by 
using  a  two-week  recall  period.   Error 
for  the  one-month  recall  period  is 
usually  an  overestimate  of  one  visit 
suggesting  telescoping.   This  problem 
can  also  be  improved  by  the  use  of 
bounding  procedures.   Error  for  the 
three-month  period  is  slightly  more 
often  (53.8  percent),  an  underestimate 
suggesting  some  recall  decay. 

Hospitalizations.   The  second 
utilization  question  asked,  pertains  to 
overnight  hospitalizations  and  reads 
"Since  July  1,  1992  a  year  ago,  were  you 
a  patient  in  a  hospital  OVERNIGHT?" 
Hospitalizations  are  large  events 
subject  to  telescoping  in  survey 
reporting.   As  Table  3  indicates,  only 
63.3  percent  of  the  adult 
hospitalizations  reports  for  yes/no 
responses  matched  records  data.   Most  of 
this  error  is  due  to  overestimating, 
suggesting  the  effect  of  telescoping  in 
hospitalizations  from  a  previous  period 
or  to  less  likely  record  error  (some 
reports  may  be  missing  due  to  unexpected 
lags  in  billing) .   For  those  reporting 
"yes",  they  then  answer  the  question, 
"How  many  different  times  did  you  stay 
in  any  hospital  overnight  or  longer 
since  July  1,  1992  a  year  ago?"   As 
Table  3  indicates,  84.1  percent  have 
exact  matches  on  the  actual  number  of 
hospital  stays.   Of  these,  all  errors 
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were  either  missing  one  (8.7  percent)  or 
two  (7.2  percent)  overnight  stays. 
Therefore,  getting  more  accurate 
estimates  for  adult  hospitalizations 
requires  increasing  accuracy  at  the 
"yes"  or  "no"  question.   Perhaps  asking 
the  number  of  hospitalizations  directly 
over  the  last  year  without  asking  a 
yes/no  question  first  would  in  itself 
improve  response  accuracy. 

For  children,  the  accuracy  in  the 
yes/no  response  to  the  first  question  on 
whether  hospitalized  is  almost  equal  to 
that  for  adults  at  63.2  percent.   Once 
again,  most  err  in  the  direction  of 
saying  "yes"  when  the  records  suggest 
the  correct  answer  is  "no".   The 
percentage  of  matches  for  the  number  of 
hospitalizations  reported  for  children 
is  83.3  percent:  again  almost  equally 
accurate  to  that  of  adults.   When  there 
is  error  in  reporting  the  number  of 
hospitalizations  for  children,  most  are 
underestimates  off  by  one  less  overnight 
stay  than  the  records  indicate. 
Although  this  would  be  unusual,  there 
may  be  an  unexpected  lag  in  the  billing 
or  reporting  in  the  records  accounting 
for  this  difference.   Hospitalizations 
are  large,  infrequent  events  and  for 
these  the  accuracy  is  more  equivalent 
for  both  adult  self -reports  and  adults 
reports  for  a  child.   For  both  adults 
and  children,  the  clear  majority  (88% 
for  children,  87%  for  adults)  reported 
only  1  overnight  stay  in  the  hospital 
over  the  previous  year.9 

Questions  during  the  hospitalizations 
were  also  asked  as  follows:  "Since  July 
1,  1992  a  year  ago,  on  what  date  did  you 
enter  the  hospital  the  last  time?   The 
time  before  that?"   A  high  94.8  percent 
of  the  month  of  hospitalization  matched 
for  adults.   The  match  for  the  full  date 
which  included  the  day  was  70.7  percent. 
The  correct  match  for  the  number  of 
nights  of  hospital  stays  was  56.9 
percent  for  exact  match  and  8  6.2  percent 
if  a  match  is  defined  as  ±1  night.   The 
survey  question  for  this  reads,  "For  the 
stay  beginning  (READ  DATE) ,  how  many 
nights  were  you  in  the  hospital?"   The 
respondents  interpretation  or  perception 
of  days  verses  nights  in  the  hospital 
may  be  the  source  of  error  for  this 
question.   If  they  were  in  the  hospital 
for  a  substantial  part  of  a  day,  they 
might  want  to  report  this  as  an 
overnight's  stay.   In  other  words,  the 
respondent  may  want  to  get  credit  for 
staying  in  the  hospital  and  want  to 
actually  report  "days"  in  the  hospital. 

There  was  an  approximately  equal 
percentage  of  matches  on  month  of 
hospitalizations  for  children  as  adults 
at  9  3.3  percent.   There  was  a  much 
better  match  on  the  full  date  of 
hospitalization  for  children  at  90 
percent.   The  exact  match  on  number  of 
nights  stay  for  children  was  53.3 
percent  (slightly  less  than  that  for 


adults)  or  83.3  percent,  if  the  ±1  night 
criterion  for  match  is  used. 

Conclusion 

The  survey  literature  supports  the 
premise  that  eligibility  and  health 
utilization  measures  will  be  subject  to 
recall  bias  in  respondent  reporting  in 
any  population,  including  a  poorer,  less 
educated  one.   In  fact,  the  literature 
suggests  the  recall  error  increases 
proportionately  with  the  length  of  the 
recall  period  for  such  measures.   The 
findings  from  this  record  check  study 
support  these  premises  for  utilization 
measures.   However,  error  in  respondent 
reporting  for  eligibility  status  is 
better  explained  by  the  lack  of 
awareness  of  their  status  during  a  one 
to  two  month  transition  period  rather 
than  to  recall  bias.   Certain  measures 
of  eligibility  and  utilization  data  are 
more  critical  than  others  to  a  national 
survey  of  Medicaid  beneficiaries  and 
accuracy  in  these  measures  should  be 
emphasized. 

Whether  the  respondent  was  actively 
on  Medicaid  over  the  entire  12 -month 
period  is  critical  information.   The 
match  of  survey  to  records  data  is  much 
higher  than  the  noncritical  for  this 
critical  information  at  91.8  percent  for 
adult  report  and  89  percent  for  the 
child  report.   However,  this  information 
is  so  critical  that  it  needs  to  be  even 
closer  to  a  100  percent  match  to  dismiss 
the  problem.   A  subsequent  analysis 
indicated  that  mismatch  in  survey 
response  error  was  largely  due  to  a 
transition  month  phenomenon  which 
resulted  in  lack  of  awareness  of  the 
respondent  as  to  their  Medicaid  status. 
Recall  bias  was  shown  to  be  less  of  a 
problem  for  respondent  reports  of 
eligibility  for  both  the  adult  and 
child.   Therefore,  accuracy  in 
respondent  reports  of  eligibility  may  be 
sufficient  to  support  the  purposes  of 
the  national  survey,  but  better  still  is 
to  use  eligibility  data  from  records  to 
supplement  survey  data. 

Depending  on  the  emphasis  on  the 
national  survey,  precise  utilization 
measures  may  be  less  critical  than  this 
eligibility  information.   The  one-month 
recall  period  for  doctor's  visits  was  a 
better  match  (more  accurate)  for  both 
adult  and  child  reports,  but 
substantially  better  for  adult  reports. 
If  the  match  is  defined  as  ±  1  visit, 
the  match  is  improved  dramatically, 
especially  for  child  reports.   The 
three-month  recall  period  for  doctor 
visits  is  of  poor  accuracy  for  adult 
reports  and  not  adequate  but  better  for 
the  child  reports.   The  one  month  recall 
period  introduced  telescoping  in  of 
visits,  the  three-month  period  was 
subject  to  recall  decay.   These  findings 
suggest  that  accuracy  in  reporting 
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doctor's  visits  may  be  optimum  at  a  2  - 
week  recall  period,  especially  if  used 
with  bounding  procedures  to  reduce  the 
effect  of  telescoping. 

The  reporting  for  hospitalizations  is 
good  at  83-84%  marker  for  the  number  of 
hospitalizations  occurring  during  the 
past  year.   However,  hospitalizations 
are  rare  events  (the  majority  of  adults 
and  children  only  reported  1 
hospitalization)  in  this  young 
population  and  should  be  reported  more 
accurately.   The  greatest  source  of 
error  was  in  the  initial  question  which 
asked  whether  or  not  there  was  a 
hospitalization  during  the  previous  12 
months.   Only  63  percent  of  the  adult 
and  child  reports  matched  the  record. 
Furthermore,  the  accuracy  was  highest  at 
around  93-94%  for  the  reporting  for 
month  of  occurrence.   These  findings 
indicate  that  asking  the  more  specific 
information  on  hospitalizations  first 
will  improve  the  accuracy  of  reporting. 
Certainly,  skipping  the  question  on 
whether  hospitalized  in  favor  of  asking 
the  number  of  hospitalization  should 
reduce  error  in  respondent  reports. 
Finally,  the  accuracy  of  reporting  of 
number  of  nights  stayed  was  not  good  and 
might  be  improved  by  emphasizing 
"nights"  not  "days"  in  the  hospital. 

Accuracy  of  utilization  measures  need 
to  be  improved  by  either  shortening  the 
recall  period;  using  memory  aids  and 
bounding  procedures  (a  costly 
alternative) ;  or  by  supplementing  survey 
data  with  records  data.   It  is  important 
to  keep  in  mind  that  while  these 
measures  are  subject  to  recall  bias, 
there  are  other  critical  measures  such 
as  access  to  care  or  satisfaction 
measures  which  will  be  asked  on  a 
national  survey.   Because  these  measures 
do  not  involve  precise  frequency  of 
event  reporting,  they  are  not  likely  to 
be  subject  to  the  recall  problem  to  this 
extent.   Recall  bias  is  more  problematic 
in  behavioral  frequency  questions. 


The  survey  sample  was  randomly 
assigned  to  either  a  one  or  three 
month  recall  period. 

Complete  tables  are  available  as  part 
of  the  final  report  or  upon  request. 

The  probe  read  "Include  all  types  of 
doctors,  such  as  dermatologists, 
psychiatrists  and  ophthalmologists, 
as  well  as  general  practitioners  and 
osteopaths.   Do  not  count  times  while 
an  overnight  patient  in  a  hospital. 

This  data  is  not  presented  in  a  table 
but  reported  from  a  separate  report. 

A  two-week  recall  period  was  not 
tested  as  part  of  this  study. 

Again,  this  data  is  not  reported  in  a 
table  but  elsewhere  in  a  separate 
report. 


ENDNOTES 

'•   See  "Assessment  of  Access  to  Care 
Pilot  Survey  of  Medicaid 
Beneficiaries"  by  Eisenhower  et  al . , 
Physician  Payment  Review  Commission, 
November  24,  1993. 

2   A  review  of  the  literature  is 
available  as  part  of  the  final 
report . 

3-   Bounding  involves  presenting  some 
information  gathered  from  one  period 
for  a  respondent  to  that  respondent 
for  a  subsequent  survey.   Memory  aids 
include  the  use  of  checklists, 
flashcards,  maps,  pictures,  and 
calendars . 
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TABLE  1 
MEDICAID  ELIGIBILITY  FOR  ALL  MONTHS-AUGUST  1992  TO  JULY  1993 


Adults 

Frequency 

Percent 

Cumulative  Frequency 

Cumulative  Percent 

Match  Not  Active' 

59 

20.3 

59 

20.3 

Match  Active2 

208 

71.5 

267 

91.8 

Mismatch  Survey  Active 

14 

4.8 

281 

96.6 

Mismatch  Record  Active 

10 

3.4 

291 

100.0 

Frequency  Missing  =  5 

Children 

Frequency 

Percent 

Cumulative  Frequency 

Cumulative  Percent 

Match  Not  Active 

69 

22.3 

69 

22.3 

Match  Active 

207 

66.8 

276 

89.0 

Mismatch  Survey  Active 

20 

6.5 

296 

95.5 

Mismatch  Record  Active 

14 

4.5 

310 

100.0 

Frequency  Missing  =  5 

1  This  is  a  match  where  the  respondent  said  they  were  not  actively  on  Medicaid  for  all  12  months. 

2  This  is  a  match  where  the  respondent  said  they  were  actively  on  Medicaid  for  all  12  months. 


TABLE  2 

SUMMARY  STATISTICS  ON  MIS  MATCH /MATCH  FOR  ONE  MONTH  AND  THREE 
MONTH  RECALL  OF  DOCTOR'S  VISITS  -  ADULT  AND  CHILD 

I.   ADULT  SELF-REPORT 


A.     One  Month  Recall  -  Adult 

Frequency 

Percent 

Cumulative  Frequency 

Cumulative  Percent 

Mismatch 

25 

49.0 

25 

49.0 

Match 

26 

51.0 

51 

100.0 

Magnitude  - 
Minus 
Plus 

4 
21 

16.0 
84.0 

4 

25 

16.0 
100.0 

B.     Three  Month  Recall  -  Adult 

Mismatch 

49 

73.1 

49 

73.1 

Match 

18 

26.9 

67 

100.0 

Magnitude  - 
Minus 
Plus 

27 
22 

55.1 
44.9 

27 
49 

55.1 

100.0 

II.   ADULT  REPORT  FOR  THE  CHILD 


A.     One  Month  Recall  -  Child 

Frequency 

Percent 

Cumulative  Frequency 

Cumulative  Percent 

Mismatch 

25 

42.4 

25 

42.4 

Match 

34 

57.6 

59 

100.0 

Magnitude  - 
Minus 
Plus 

6 
19 

24.0 
96.0 

6 

25 

24.0 
100.0 

B.     Three  Month  Recall  -  Adult 

Mismatch 

39 

49.4 

39 

49.4 

Match 

40 

50.6 

79 

100.0 

Magnitude  - 
Minus 
Plus 

21 
18 

53.8 
46.2 

21 

30.8 
100.0 
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TABLE  3 
A.    HOSPITALIZATION:    ANY  OVERNIGHT  STAYS  DURING  THE  PAST  YEAR  (YES/NO  RESPONSE) 


Adults 

Frequency 

Percent 

Cumulative 
Frequency 

Cumulative 
Percent 

I/P  CLAIM  NO  SURVEY 

12 

11.0 

12 

11.0 

NO  I/P  CLAIM  MATCH 

28 

25.7 

40 

36.7 

I/P  CLAIM  MATCH 

69 

63.3 

109 

100.0 

Children 

Frequency 

Percent 

Cumulative 
Frequency 

Cumulative 
Percent 

I/P  CLAIM  NO  SURVEY 

6 

10.5 

6 

10.5 

NO  I/P  CLAIM  MATCH 

15 

26.3 

21 

36.8 

I/P  CLAIM  MATCH 

36 

63.2 

57 

100.0 

B.    HOSPITALIZATIONS:    OF  THOSE  SAYING  "YES,"  DEGREE  OF  MATCH  OF  OVERNIGHT  STAYS 


Adults 

Frequency 

Percent 

Cumulative 
Frequency 

Cumulative 
Percent 

2  Less  than  record 

5 

7.2 

5 

7.2 

1  Less  than  record 

6 

8.7 

11 

15.9 

EQUAL 

58 

84.1 

69 

100.0 

Children 

Frequency 

Percent 

Cumulative 
Frequency 

Cumulative 
Percent 

1  Less  than  record 

5 

13.9 

5 

13.9 

EQUAL 

30 

83.3 

35 

97.2 

5  More  than  record 

1 

2.8 

36 

100.0 
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USE  OF  CASE  MIX  DATA  TO  MONITOR  THE  IMPACT  OF  TENNCARE 

Jim  Shmerling,  LBCMC 
Maren  Proulx 


On  January  1,  1994,  the  State  of 
Tennessee  implemented  a  dramatic 
effort  in  health  care  reform  entitled 
"TennCare."  TennCare  is  a  five-year 
demonstration  project  that  utilizes 
managed  care  organizations  to  contract 
health  care  for  the  previously 
eligible  Medicaid  population  as  well 
as  the  uninsured. 

In  addition  to  a  significant 
projected  cost  savings,  the  TennCare 
program  was  intended  to  provide 
affordable  comprehensive  coverage  for 
virtually  every  Tennessean.  It 
received  the  Health  Care  Financing 
Administration's  (HCFA)  approval  in 
November  1993  and  was  implemented  in 
January  1994. 

There  has  been  significant 
criticism  specifically  related  to  the 
underfunding  of  the  program  and  its 
hurried  implementation.  However,  to 
date  there  has  been  limited  clinical 
analysis  of  TennCare 's  impact  on  its 
intended  beneficiaries.  In  an  effort 
to  assess  the  impact  of  TennCare  on 
its  pediatric  population,  Le  Bonheur 
Children's  Medical  Center  utilized  the 
National  Association  of  Children's 
Hospitals  and  Related  Institutions' 
(NACHRI)  Case  Mix  Comparative 
Database. 

Le  Bonheur  Children's  Medical 
Center  is  a  regional  referral  center 
with  225  beds.  Its  primary  and 
secondary  service  area  includes 
portions  of  Tennessee,  Mississippi, 
Arkansas,  Missouri,  Kentucky  and 
Alabama.  More  than  100,000  children 
receive  treatment  on  an  inpatient  or 
outpatient  basis  every  year. 

NACHRI 's  Case  Mix  Database  is 
built  on  pediatric  inpatient  medical 
record  abstracts  from  over  50 
children's  hospitals,  and  hundreds  of 
teaching  and  nonteaching  facilities. 
The  medical  record  abstract  data  were 
available  by  ICD-9-CM  code  for  both 
principal  and  secondary  diagnoses. 

NACHRI  and  Le  Bonheur  limited 
their  analysis  to  the  data  of  high- 
volume  patient  populations, 
specifically  those  admitted  with  a 
respiratory  principal  diagnosis  (PDX). 
The  authors  reasoned  that  by  using 
selected  diagnoses  that  represented 
high  volumes  of  both  Medicaid  and  non- 
Medicaid  patients,  existing  trends 
could  be  identified  while  still 
identifying  external  influences,  such 
as  whether  there  was  a  light  or  severe 
year  for  RSV  respiratory  infections 
for  children.  These  data  could  then  be 
used  to  mirror  the  overall  impact  of 


TennCare  on  the  patient  population  of 
Le  Bonheur. 

The  analysis  looked  at  the 
principal  diagnosis  of  each 
respiratory  patient,  the  age  of  the 
patient,  and  whether  there  were  any 
congenital  and  chronic  conditions 
(often  reflected  through  secondary 
diagnoses).  The  analysis  included  all 
patients  with  a  respiratory  principal 
diagnosis  who  were  one  week  or  older 
at  time  of  admission.  For  all  of 
these  patient  breakouts,  the  analysis 
examined  whether  the  patient  treatment 
included  mechanical  ventilation  (an 
indication  that  the  patient  was 
severely  ill)  or  a  major  or  extreme 
complication  as  identified  through  the 
APR-DRG  classification  system. 
Finally,  for  all  these  breakouts,  the 
analysis  provided  length  of  stay  and 
cost  (operating  and  capital) 
statistics. 

NACHRI  has  developed  a  method  of 
mapping  approximately  2,000  of  the 
ICD-9-CM  codes  into  a  series  of  over 
60  congenital  and  chronic  child  health 
conditions.  The  system  was  developed 
to  track  the  small  proportion  of  the 
pediatric  population  that  is  the 
sickest:  those  who  require 
specialized  health  care  services  and 
expend  the  most  health  care  dollars. 
The  application  of  these  flags  was 
particularly  important  in  measuring 
the  Respiratory  PDX  population  before 
and  after  TennCare.  For  example,  while 
many  pediatric  patients  are  admitted 
with  pneumonia,  the  patients  with 
chronic  secondary  conditions  such  as 
bronchopulmonary  dysplasia,  congenital 
heart  anomalies,  cerebral  palsy,  HIV 
or  blood  disorders  will  require 
disproportionate  care  and  health  care 
dollars.  NACHRI 's  Congenital  and 
Chronic  flags  allowed  the  reviewers  to 
track  these  resource-intensive 
patients  whose  principal  diagnosis  may 
not  have  identified  them  as  high 
acuity. 

With  these  tools  in  hand,  NACHRI 
and  Le  Bonheur  began  a  review  of  its 
pre-  and  post-TennCare  medical 
Respiratory  PDX  admissions.  One  of 
the  principal  philosophical 
underpinnings  of  TennCare  was  to 
provide  low-income  children  access  to 
primary  and  preventive  care  prior  to 
the  point  at  which  they  would  require 
hospitalization.  The  authors  sought 
to  determine  the  impact  that 
introduction  of  TennCare  in  1994  had 
on  admissions  of  Respiratory  PDX 
patients  comparing  1992  and  1993  data 
with  1994. 


261 


They  found  that  in  1994,  there 
was  a  significant  decrease  in  the 
overall  volume  of  Respiratory  PDX  from 
the  average  of  the  two  years  prior. 
The  total  decrease  was  approximately 
27  percent  for  Medicaid  and  46  percent 
for  non-Medicaid  patients,  suggesting 
that  1994  was  not  a  high-volume 
respiratory  infectious  disease  year 
for  the  Memphis  area.  The  case  volume 
for  Asthma  and  Pneumonia  NOS  (not 
otherwise  specified),  representing 
over  60%  of  the  total  Respiratory  PDX, 
decreased  for  the  Medicaid  population 
14.5%  and  8.4%  respectively  while  the 
non-Medicaid  population  decreased 
39.5%  and  3  5%  respectively.  The 
number  of  admissions  for 
Bronchiolitis,  most  often  caused  by 
RSV  infection,  decreased  by  more  than 
50%. 

Among  those  medical  inpatients 
with  Respiratory  PDX,  there  was  a  20% 
increase  in  admissions  of  TennCare 
children  with  chronic  respiratory 
conditions.  Hospitalized  patients 
with  a  Respiratory  PDX  who  have  "other 
congenital  and  chronic"  conditions 
remained  the  same  for  Medicaid 
patients.  The  pattern  for  non- 
Medicaid  patients  is  different  in  both 
instances  showing  significant 
decreases  in  respiratory  admissions 
for  chronically  ill  children. 

The  most  dramatic  change  in  the 
first  year  of  TennCare  was  in  the 
length  of  stay  and  average  operating 
costs  for  this  population.  Medicaid 
Respiratory  PDX  patients  with 
congenital  and  chronic  conditions 
experienced  a  sharp  increase  in  costs 
associated  with  their  care  along  with 
increases  though  less  pronounced  in 
the  average  length  of  stay.  Costs  for 
TennCare  patients  with  chronic 
respiratory  conditions  nearly  doubled 
from  the  average  of  the  same  Medicaid 
patients  of  the  prior  two  years. 
Costs  for  TennCare  patients  with  other 
congenital  and  chronic  conditions 
increased  by  20%  in  the  year  after 
TennCare.  In  contrast,  non-Medicaid 
patients  demonstrated  a  decline  in 
both  length  of  stay  and  costs 
associated  with  the  care  of 
respiratory  patients  who  have 
congenital  and  chronic  conditions. 

The  authors  concluded  that 
TennCare  did  not  appear  to  have  a 
positive  impact  in  promoting  the 
health  of  children  particularly  those 
with  chronic  and  congenital 
respiratory  conditions.  In  fact, 
hospital  admissions  and  length  of 
stays  for  these  at-risk  children 
increased  and  were  more  prolonged 
suggesting  a  higher  acuity  and 
corresponding  higher  costs  for  their 
care.  This  phenomenon  occurred  in  a 
period  of  time  when  there  was  an 
overall  decline  in  the  Respiratory  PDX 
population  admitted  to  the  Hospital 


and  when  the  non-TennCare  patients 
actually  experienced  a  lower 
utilization  of  services. 

Additional  studies  of  these 
populations  are  anticipated.  The  Case 
Mix  Data  System  will  enable  Le  Bonheur 
to  conduct  control  studies  of  its  most 
at-risk  patients.  Medicaid  managed 
care  is  being  proposed  as  an  integral 
component  to  health  care  reform. 
However,  its  impact  particularly  on 
vulnerable  children  is  not  known.  As 
this  study  indicates,  a  Medicaid 
managed  care  product  may  not  reduce 
health  care  costs  or  improve  the 
health  status  of  pediatric  enrol lees. 
Because  a  primary  motivation  for 
Medicaid  managed  care  is  cost 
containment,  additional  research  is 
imperative  to  identify  the  clinical 
impact  as  well  as  the  financial. 
NACHRI's  APR-DRG  classification 
system  and  its  classification  of 
congenital  and  chronic  conditions 
provides  the  means  to  evaluate  changes 
in  inpatient  case  mix.  The  Case  Mix 
Data  System  will  serve  as  the  source 
of  data  for  future  analysis  of 
TennCare 's  impact  on  patients  served 
by  Le  Bonheur.  In  addition,  because 
there  are  several  other  children's 
hospitals  in  Tennessee  who  also 
participate  in  the  Case  Mix  System, 
comparative  studies  can  be  conducted 
on  a  state-wide  basis. 


RESPIRATORY    PDX 


Age  >  1  Week  Old 

Congenital  and  Chronic  Conditions, 
e.g.  : 

-  Bronchopulmonary  Dysplasia 

-  Congenital  Heart  Anomalies 

-  Cerebral   Palsy 

-  HIV 

-  Other  Blood  Disorders 

Mechanical  Ventilation 


Major  or  Extreme  Complication  per 
APR-DRGs 


Length  of  Stay 


Costs 
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CASES  of  inpatients 
with  a  Medical  RESPIRATORY  PDX 
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AVERAGE  OPERATING  COSTS 

for  All  Inpatients  with  a  Medical 

RESPIRATORY  PDX 
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20TH  CENTURY  ALCHEMISTS: 
GENERATING  RESEARCH  DATA  FROM  A  BILLING  DATABASE 

Lynne  Fullerton,  University  of  New  Mexico 
M.  Jack  Lee,  Lenora  Olson,  David  Sklar 


Introduction 

The  appropriateness  of  health  care 
utilization,  and  the  cost  of  emergency 
care,  is  of  concern  in  a  time  of  health 
reform.  One  aspect  of  emergency 
services,  prehospital  care,  has  only 
recently  become  the  focus  of  research 
related  to  health  care  utilization.  The 
only  national,  industry-specific  journal 
dedicated  to  prehospital  research  is 
Prehospital   and  Disaster  Medicine, 
first  published  in  1985.  Our 
understanding  of  prehospital  medicine  is 
also  limited  by  the  fact  that 
prehospital  medicine  has  changed 
considerably  in  the  past  three  decades. 
EMS  was  initially  promoted  and  developed 
to  treat  and  transport  critically  ill 
patients,  such  as  cardiac  arrest 
patients  and  those  with  potentially 
fatal  trauma,  to  emergency 
departments  .2'3,4  Health  care  needs  have 
changed  such  that  prehospital  care  is 
now  less  often  involved  in  the  treatment 
of  fatally  ill  or  injured  persons,  and 
more  often  involved  in  the  treatment  of 
patients  for  whom  emergency  department 
care  may  not  be  required. 

Injury  data  sources  available  at 
national  and  local  levels  focus 
primarily  on  mortality  statistics,  data 
available  from  police  reports,  and 
injuries  associated  with  particular 
injury  sites  (e.g.,  the  National  Head 
and  Neck  Injury  Registry)  or  mechanisms 
(e.g.,  the  Boating  Accident  Reporting 
System) .   Few  data  sources  exist  to 
examine  non-fatal  injuries,  and  it  is 
particularly  difficult  to  follow  the 
course  of  an  injury  from  prehospital 
services  through  the  ultimate  outcome. 


There  are  currently  no  national 
data  sources  available  to  evaluate  the 
use  of  prehospital  services  for 
emergency  care  and  utilization  patterns. 
The  National  Uniform  Data  Set  for  EMS, 
under  the  auspices  of  the  National 
Highway  Traffic  Safety  Association 
(NHTSA) ,  proposes  to  begin  to  address 
this  need.  This  data  set  is  being 
developed  to  collect  national  injury 
data  and  related  EMS  call  information  to 
aid  research  in  traffic-related 
injuries.  This  database  is  an  example  of 
how  injury  tracking  may  benefit  from  the 
use  of  prehospital  data. 

Prehospital  data  provide  an 
epidemiologic  window  onto  community 
patterns  of  non-fatal  injury  and  illness 
for  which  ambulance  transport  occurs. 
Whereas  specific  hospital  data  may  be 
biased  by  payer  class,  location,  or 
expertise  of  the  hospital,  data  from 
ambulance  transports  to  multiple 
hospitals  provide  an  overall  picture  of 
certain  medical  problems  in  a  community. 
Ambulance  data  have  traditionally  been 
utilized  for  quality  assurance  and 
billing  purposes,  and  are  a  relatively 
untapped  resource  in  disease  and  injury 
research.  We  attempted  to  use  these  data 
to  provide  epidemiologic  information 
concerning  medical  conditions  that  have 
not  previously  been  well  defined  from  a 
community  context . 

To  examine  prehospital  care  in  a 
large,  urban  community,  we  used  data 
collected  by  the  primary  ambulance 
service  for  this  community.  These  data 
can  be  used  to  examine  ambulance 
utilization  patterns  over  time,  and  to 
suggest  alternatives  to  possibly 
unnecessary  ambulance  use. 


267 


Methods 


We  used  a  propriety  billing 
software  (MedOccur™)  ,  an  AMOS 
METROPOLIS™  database,  with  the 
following  features:  built  in  pre- 
structured  reports;  no  flat  file  or 
relational  capacity;  and,  no  ad  hoc 
querying  or  reporting  capacity.  From 
this  database,  we  created  an  artificial 
relational  database  by  downloading 
hardwired  reports,  limiting  the  number 
of  records  by  downloading  records 
restricted  by  parameters  such  as  year  of 
transport  and  diagnosis. 

The  steps  in  the  translation 
process  were  as  follows:  1)  we  printed 
the  MedOccur™  reports  to  a  spooler 
file,  creating  an  ASCII  file;  2)  using 
an  AlphaMicro™  interface,  we  downloaded 
the  ASCII  file  to  a  personal  computer; 
3)  using  a  spreadsheet  program,  we 
imported  the  ASCII  file  and  converted  it 
to  a  database  program;  and  4)  we 
manipulated  the  data  and  translated 
various  coded  fields  to  create  a 
database  compatible  with  research. 


patients  who  have  no  other  emergent 
conditions.  These  data  may  also  be  used 
to  study  the  cost-effectiveness  of 
constructing  and  staffing  a 
detoxification/alcohol  treatment  center 
in  our  community. 


Fig.  1:  Percentage  of  Patients  In  Each  Diagnostic  Category  Who 
Were  Transported  Two  Or  More  Times  For  The  Same  Chief 
Complaint,  AAS  data,  1992-1994 
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Results 

Since  the  inception  of  our 
collaboration,  we  have  developed  several 
databases  to  study  a  variety  of 
prehospital  issues.  In  our  first 
project,  we  compared  numbers  of 
diagnosis-specific  transports  among 
patients  transported  for  acute  alcohol 
intoxication  (EtOH) ,  seizures  (Sz) ,  and 
asthma/shortness  of  breath  (SOB) .  We 
found  that  the  proportion  of  patients 
transported  two  or  more  times  for  the 
same  chief  complaint  was  highest  among 
EtOH  patients,  among  whom  nearly  half 
(49%)  had  been  transported  more  than 
once,  contrasted  with  21%  of  Sz  patients 
and  14%  of  SOB  patients  (p  <  .0001) 
(Figure  1) .  The  results  of  this  study 
will  be  used  to  examine  the  efficacy  of 
alternative  prehospital  treatment 
protocols  for  acutely  intoxicated 


In  our  second  project,  we  examined 
the  association  between  ambulance  runs 
for  acute  alcohol  intoxication  (EtOH) 
and  ambulance  runs  for  assault .  We  found 
that  the  proportion  of  EtOH  patients  who 
also  had  assault  transports  increased 
with  increasing  frequencies  of  EtOH 
transports  (15=4.9;  R2  =  .92)  (Figure  2). 
Our  finding  that  over  half  (56.6%)  of 
patients  transported  ten  or  more  times 
for  EtOH  had  been  transported  on  another 
occasion  for  assault  has  clear 
implications  for  violence  prevention 
interventions . 
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Fig.  2:  Percentage  for  EtOH  Patients  Transported  with  a  Chief  complaint  of  Assault 
as  a  Function  of  Number  of  EtOH  Transports,  AAS  data,  1992-1994 
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In  addition  to  injury  research,  we 
have  also  collaborated  on  a  project 
evaluating  response  times  to  cardiac 
arrest  calls  by  response  levels,  using 
the  Emergency  Medical  Priority  Dispatch 
System  (EMPDS) .  Using  1994  cardiac 
arrest  data,  we  found  that  the  large 
majority  of  cases  were  dispatched  with 
the  highest  level  of  response  (n=284; 
93.1%)  .  Our  study  demonstrated  that  the 
EMPDS  is  highly  accurate  with  regard  to 
cardiac  arrests.  Future  research  will 
evaluate  the  EMPDS  with  respect  to 
outcomes  other  than  cardiac  arrests. 

Discussion 

The  most  important  result  of  our 
collaboration  was  the  development  of  an 
ongoing  research  group  dedicated  to 
examining  issues  related  to  public 
health  and  prehospital  medicine.  This 
project  combined  the  expertise  of  a 
medical  school  department  of  emergency 
medicine  and  the  primary  ambulance 
service  for  the  area  to  design  and 
conduct  multiple  projects  relevant  to 
prehospital  research.  Our  projects  are 
unusual  in  that  they  provide  an  insight 
into  nonfatal  injuries  and  disease  in 
the  community. 


The  results  of  our  research  have 
many  potential  applications.  One  use  of 
our  collaborative  projects  concerns 
quality  control  and  evaluation  of 
existing  prehospital  protocols.  Other 
prehospital  research  has  been  used  to 
evaluate  the  utility  of  traditional 
prehospital  interventions  such  as 
treatment  of  hyperventilation;  the 
results  of  such  research  have  indicated 
that  the  potential  consequences  of  this 
intervention  may  exceed  the  benefits. 
Similarly,  our  data  have  been  used  to 
evaluate  response  times  to  cardiac 
arrests  cases  associated  with  EMPDS 
response  levels.  Our  study  of  repeated 
ambulance  use  will  be  expanded  and  used 
to  suggest  new  trends  in  prehospital 
care  such  as  treat-and-ref er,  rather 
than  the  current  method  of  treat-and- 
transport .  Such  changes  have  the 
potential  to  increase  the  efficacy  of 
prehospital  care,  while  at  the  same  time 
reducing  the  costs  associated  with 
unnecessary  use  of  emergency  rooms. 
These  projects  provide  us  with 
information  we  can  use  to  advocate  for 
changes  in  medical  services  in  the 
community  as  has  been  done  previously  by 
groups  promoting  EMS  in  primary  health 
care.8  One  example  of  collaboration 
involving  such  groups  is  the  annual 
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Sandkey  conference  which  focuses  on  the 
role  of  EMS  in  the  delivery  of  primary 
health  care.  '   The  Sandkey  conferences 
have  included  individuals  from  several 
disciplines,  including  EMS,  nursing,  and 
medicine.  Recommendations  resulting  from 
Sandkey  conferences  have  included:  1) 
customizing  programs  to  build  alliances 
and  collect  health  data;  2)  adding/ 
increasing  components  such  as  disease 
prevention  and  health  education  to  EMS; 
and  3)  using  EMS  for  home  health  care 
delivery,  including  assessment, 
diagnosis,  and  follow-up.  Another 
recommendation  was  to  develop  treat -and- 
refer  protocols  to  supplement  the 
currently  used  treat -and- transport 
protocols . 

Our  collaboration  has  benefited 
both  organizations  involved  in  the 
research.  The  ambulance  service  now  has 
the  necessary  data  to  evaluate  the 
efficacy  and  cost  effectiveness  of 
various  EMS  treatment  interventions  and 
alternatives.  Our  research  has  also 
prompted  the  ambulance  service  to 
investigate  available  software  that  will 
permit  development  of  customized  data 
queries  and  reports.  The  use  of  this 
software  will  obviate  the  need  to  have 
reports  generated  within  the  current 
operating  system,  ultimately  saving 
hundreds  of  dollars  per  report.  The 
emergency  department  has  benefited  in 
terms  of  access  to  a  powerful  and  rich 
source  of  data  for  faculty  and 
residents'  research  and  community 
projects.  These  data  have  provided 
baseline  and  long-term  information  on 
injury  morbidity  for  planning  effective 
injury  prevention  initiatives. 

Currently,  research  such  as  ours  is 
the  only  way  to  access  community  level 
non-fatal  injury  data.  However,  efforts 
are  underway  both  nationally  and  in  New 
Mexico  to  link  databases  from  multiple 
sources  relevant  to  injury  research.  We 
anticipate  that  the  Crash  Outcome  Data 
Evaluation  System  (CODES)  projects 


initiated  by  NHTSA  will  begin  to  address 
the  need  for  non-fatal  injury  data  in 
the  nation. 11,12'13  The  purpose  of  the 
CODES  projects  is  to  link  injury  data 
from  police  records,  emergency 
departments,  coroners,  and  hospital 
discharge  data  to  obtain  a  complete 
picture  of  the  injured  patient  in  our 
health  care  systems.  CODES  projects  are 
currently  operating  in  several  states, 
and  already  have  provided  excellent 
results.  In  New  Mexico,  the  data  linkage 
system  is  still  in  the  process  of 
development,  and  it  is  not  expected  that 
data  will  be  available  from  the  New 
Mexico  CODES  project  until  1997. 14  In 
the  meantime,  data  from  projects  such  as 
ours  can  be  used  to  address  the  need  for 
information  concerning  non-fatal  injury 
in  our  state. 

We  have  encountered  limitations  in 
the  use  of  software  not  originally 
designed  for  research.  The  use  of  a  non- 
industry  standard  operating  system  was 
associated  with  limited  technical 
support .  The  software  does  not  support 
standard  querying  language  (SQL) 
queries,  and  as  such  we  were  limited  to 
the  customized  reports  prepared  by  the 
software  vendor  for  the  purposes  of 
billing.  Finally,  the  software  does  not 
create  a  relational  file  type  database, 
which  made  it  difficult  to  generate 
databases  with  fields  located  in  more 
than  one  report.  These  limitations  are 
being  addressed  by  the  introduction  of 
software  designed  to  interface  with  the 
AMOS  METROPOLIS™  database  and  permit 
development  of  customized  queries  and 
reports. 

We  also  found  sources  of  error. 
Selection  of  ICD9  codes  by  paramedics 
was  influenced  by  use  of  a  "cheat  sheet" 
to  fill  out  the  ambulance  run  form,  i.e. 
a  reference  sheet  that  lists  the  most 
common  ICD9  codes  and  abbreviated 
descriptions  of  the  complaints 
associated  with  these  codes.-  We  found 
that  the  codes  on  the  reference  sheet 
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were  used  preferentially  by  some 
personnel,  even  when  other  ICD9  codes 
might  have  been  more  appropriate.  This 
may  have  biased  some  of  our  findings 
related  to  diagnosis-specific  outcomes. 
There  were  also  some  errors  associated 
with  manual  data  entry.  We  plan  to 
address  these  sources  of  error  through 
the  introduction  of  computerized 
charting,  to  be  introduced  within  the 
next  year  at  the  ambulance  service.  This 
technology  permits  the  paramedics  to 
enter  the  patient's  complaint  in  the 
computer,  and  the  associated  ICD9  code 
is  automatically  entered.  This  system 
will  eliminate  the  need  for  both  the 
cheat  sheet  and  the  manual  data  entry. 

Conclusion 

Our  research  group  provides  us  with 
an  ongoing  collaboration  that  allows  us 
to  investigate  topics  of  interest  to 
both  organizations.  The  first  year  of 
our  work  together  has  generated  multiple 
databases  allowing  us  to  examine  myriad 
prehospital  issues.  We  have  developed 
software  translation  protocols  and 
programs  that  will  facilitate  the 
creation  of  future  research  databases. 
In  addition,  we  are  investigating 
available  software  that  will  permit  us 
to  investigate  a  broader  range  of 
prehospital  issues  by  allowing  us  to 
incorporate  variables  from  all  reports 
into  one  database. 

This  research  has  demonstrated  that 
it  is  feasible  to  utilize  a  database 
developed  and  maintained  for  billing 
purposes  to  create  databases  amenable  to 
prehospital  research.  The  availability 
of  these  data  will  permit  both 
retrospective  and  prospective  study 
designs  at  a  moderate  cost.  The 
resulting  data  and  collaborative  effort 
can  be  used  to  guide  future  decisions 
about  prehospital  health  care  delivery 
in  our  community. 
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THE  COST  EFFECTIVENESS  OF  LINKED  STATE  DATA  FOR  HIGHWAY  SAFETY 

Dennis  E.  Utter 
National  Highway  Traffic  Safety  Administration 


By  being  at  this  conference,  I  think 
we  all  are  demonstrating  that  we  believe 
data  are  necessary  for  making  sound 
decisions.   Policy  makers  and  program 
managers  need  quality  data  to  decide  which 
programs  are  needed  and  where  they  need  to 
be  implemented.   After  implementation, 
programs  need  to  be  evaluated  to  determine 
if  the  desired  effect  was  achieved. 

The  need  for  data  is  especially  true 
in  administering  highway  safety  programs. 
Each  year,  millions  of  dollars  are  spent 
trying  to  convince  people  to  wear  safety 
belts,  use  motorcycle  helmets,  cross 
streets  at  cross  walks,  not  to  drink  and 
then  drive,  drive  safely  through 
construction  zones,  obey  speed  limits  and 
traffic  signals,  and  on  and  on. 

The  need  for  programs  like  these  is 
demonstrated  by  data  that  provide  us  with 
the  knowledge  that  motor  vehicle  crashes  on 
the  nation' s  highways  take  a  staggering 
toll  each  year: 

•  More  than  40, 000  people  are 
killed; 

•  More  than  3,000,000  people  are 
injured,  400,000  severely; 

•  Motor  vehicle  traffic  crashes 
are  the  largest  single  cause  of 
injury  and  death  for  persons 
under  the  age  of  40;  and, 

•  More  than  $137  billion  in 
economic  costs. 

The  toll  in  pain  and  suffering  and  in 
economic  loss  due  to  motor  vehicle  crashes 
is  large.   Reducing  this  toll  is  a  national 
priority. 

Despite  the  availability  of  these 
overall  numbers,  the  data  which  a  highway 
safety  analyst  at  the  state  or  local  level 
can  use  are  often  extremely  limited. 
Decisions  upon  which  at  risk  populations 
are  identified,  countermeasures  are 
developed,  and  safety  program  dollars  are 
spent  are  often  based  upon  feel  rather  than 
factual  evidence.   Evaluation  of  program 
effectiveness  is  as  often  based  on  the 
number  of  brochures  handed  out,  or 
television  spots  made,  as  on  whether  the 
crash  or  injury  rate  declined  or  the  cost 
of  hospital  care  for  injured  victims  was 
reduced  as  a  result  of  the  program. 

The  primary  highway  safety  database 
available  in  most  states  is  an  electronic 
data  file  compiled  from  information 
collected  at  the  crash  scene  by  the 
investigating  police  officer.   Every  state 
requires  that  a  report  be  filed  when  a 
crash  occurs,  if  some  minimum  amount  of 
property  damage  resulted  from  a  crash  or  if 
an  involved  person  was  injured.   These 
reports  are  collected  by  a  central  agency 
in  the  state  which  then  creates  a 
statewide,  electronic  database. 

These  statewide  police  reported  crash 
data  are  an  excellent  source  for  many 


highway  safety  needs.   They  provide 
statewide  characteristics  about  motor 
vehicle  traffic  crashes,  the  vehicles 
involved  in  them,  and,  to  some  extent,  the 
people  involved.   From  this  file,  highway 
safety  analysts  can  obtain  information 
about  the  number  of  crashes,  identify  where 
they  occurred,  and  count  the  number  of 
people  killed  and  injured  for  different 
crash  configurations  and  vehicle  types. 
Data  from  police  crash  files  on  the  number 
and  characteristics  of  fatal  crashes  have 
beer,  used  by  states,  and  by  NHTSA,  for  many 
years  to  identify  and  evaluate  highway 
safety  countermeasures.   In  fact,  highway 
safety  programs  have  concentrated  on 
reducing  fatalities.   Safety  programs  and 
motor  vehicle  crashworthiness  improvements 
have  steadily  reduced  the  crash  fatality 
rate.   People  are  now  surviving  crashes  who 
would  have  been  killed  15  or  20  years  ago. 
Consequently,  safety  programs  are  beginning 
to  emphasize  injury  prevention  and 
reduction  and  health  care  cost  reduction. 

This  is  where  police  crash  data  begin 
to  fall  short.   Police  data  lack  a 
description  of  the  injuries,  and  their 
consequences,  that  the  people  involved  may 
have  received.   Police  reports  can  tell  us 
if  an  involved  person  was  killed.   Most 
reports  provide  a  classification  for  how 
severely  a  victim  is  injured,  i.e., 
seriously  injured,  slightly  injured,  or  not 
injured.  Even  these  classifications  are 
often  inaccurate  and  vary  between  and  among 
police  jurisdictions.   And,  because  the 
police  officer  is  not  a  diagnostician, 
there  is  no  way  for  him  or  her  to  know  what 
particular  injuries  a  victim  sustained  or 
what  it  cost  to  treat  those  injuries.   For 
highway  safety  program  identification  and 
evaluation  to  be  effective  in  injury 
control  and  cost  reduction,  better 
information  on  the  types,  the  severity,  the 
treatments,  and  the  cost  of  the  injuries 
sustained  by  those  who  survive  a  motor 
vehicle  crash  is  needed. 

The  collection  of  information  is  not 
a  trivial  matter,  either  in  design  or  in 
cost.   To  establish  a  new  system  to  obtain 
information  about  the  medical  outcomes  of 
persons  injured  in  crashes  would  not  be 
likely  in  this  era  of  declining  resources. 
But,  we  at  NHTSA  believe  this  information 
can  be  obtained  in  an  efficient  and  cost 
effective  manner.   Patient  specific  data 
are  being  collected  by  different  types  of 
health  care  providers  for  numerous  other 
reasons.   If  data  files  created  from  these 
sources  could  be  linked  to  the  occupant 
specific  police  reported  crash  data, 
medical  outcome  data  could  then  be 
associated  economically  with  specific  crash 
data . 

NHTSA  has  been  exploring  this  concept 
since  the  late  1980s.   In  the  Sensitivity 
Index  Project,  the  states  of  Maine  and 
Missouri  were  able  to  link  their  crash, 
Emergency  Medical  Services  reports,  and 
hospital  discharge  summary  files  to 
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evaluate  EMS  performance.   The  linkage 
methodology  employed  in  these  projects, 
however,  was  labor  intensive  and  time 
consuming  because  it  required  exact 
matches.   Such  a  procedure  would  be 
impractical  on  a  large  scale. 

In  1991  NHTSA  was  provided  with  an 
opportunity  to  improve  on  the  file  linkage 
technology  available  to  the  states.   The 
Intermodal  Surface  Transportation 
Efficiency  Act  of  1991  (ISTEA)  instructed 
NHTSA  to  conduct  a  study  using  state  data 
to  evaluate  the  benefits  of  safety  belts 
and  motorcycle  helmets  in  terms  of 
mortality  morbidity,  severity  and  costs. 
To  conduct  such  a  study  using  state  data, 
NHTSA  concluded  that  the  data  needed  to  be 
statewide  and,  therefore,  would  bs  based  on 
police  crash  data.   Because  data  describing 
the  injury  severity  and  treatment  costs  of 
the  injured  people  were  necessary,  the  only 
way  to  obtain  these  data  was  to  link  the 
crash  data  to  available  medical  outcome 
data  files. 

In  1992  NHTSA  funded  seven  states  to 
link  their  crash  data  files  with  medical 
outcome  data  files.     NHTSA  entitled  this 
project  the  Crash  Outcome  Data  Evaluation 
System  (CODES)  project.   Exhibit  1  depicts 
the  data  files  which  could  be  linked  in  a 
CODES  project. 


iTraffic  Records 

Motor 
Vehicle 
Cnuk 

EMS 

— 

Emergency 
Department 

Hospital 
Discharge 

Rehab/ 
Long 
Term 
Care 

1 

Outpatient 

Insurance 
Claims 

Death 
Certificate 

Exhibit  1.    Crash  and  Injury  Data  Sources. 


The  files  available  in  most  of  the 
CODES  states  were  the  emergency  medical 
services,  hospital  discharge,  long  term 
care  and  rehabilitative  care  data  files. 
Three  of  the  states  had  emergency 
department  data  files  and  four  linked  to 
insurance  claims  databases.   Even  though 
each  of  the  seven  states  differed  in  which 
data  files  were  available,  they  all  were 
able  to  link  and  obtain  results  which 
supported  the  required  analysis 

NHTSA  elected  to  have  the  states  in 
the  CODES  project  use  probabilistic 
linkage,  a  procedure  which  was  first 
proposed  by  Fellegi  and  Sunter  in  1969. 


The  computer  software  developed  using  this 
procedure  mitigated  the  problem  of  dealing 
with  large  file  sizes. 

The  seven  states  were  successful  in 
linking  their  available  databases.   They 
have  provided  their  linked  data  to  NHTSA 
and  performed  analyses  which  have 
contributed  to  a  draft  Report  to  Congress 
on  the  Benefits  of  Safety  Belts  and 
Motorcycle  Helmet  Use  which  is  available  to 
the  general  public.   I'll  briefly  summarize 
some  of  the  results  of  this  study. 

NHTSA  chose  to  evaluate  the  benefits 
of  safety  belts  and  helmets  in  terms  of 
effectiveness.   Effectiveness  is  defined  as 
the  percentage  reduction  in  injuries  or 
deaths  for  people  wearing  safety  belts  or 
using  helmets.   For  example,  if  belts  are 
35  percent  effective  in  reducing 
fatalities,  35  percent  of  the  persons  who 
were  killed  while  not  wearing  belts  would 
not  have  been  killed  had  they  been  wearing 
a  safety  belt.   An  overall  estimate  of 
effectiveness  was  obtained  by  statistically 
combining  the  results  of  logistic 
regression  analyses  conducted  by  the  CODES 
states.   Exhibit  2  summarizes  the 
effectiveness  of  safety  belts  and  of 
motorcycle  helmets  by  severity  of  outcome. 


Exhibit   2. 

Effectiveness    of    Safety  Belts    and 

Motorcycle   Helmets   by  Outcome  Measure 

for   Crash- Involved  Drivers    and 
Motorcycle   Riders    in    the   CODES    States 

Outcome 
Measure 

Effectiveness 

Safety 
Belts 

Motorcycle 
Helmets 

Died 

89% 

33% 

Died   or 
Inpatient 

75% 

24% 

Died, 

Inpatient,    or 
Transported 

54% 

23% 

Any    Injury 

50% 

5% 

These  overall  results  confirmed 
previous  NHTSA  analyses  that  safety  belts 
and  motorcycle  helmets  are  effective  in 
reducing  mortality  and  that  safety  belts 
are  effective  in  reducing  morbidity. 
Although  motorcycle  helmets  are  less 
effective  at  reducing  morbidity,  keep  in 
mind  that  they  were  designed  to  protect  the 
brain  and  not  prevent  most  of  the  other 
types  of  injuries  that  a  motorcycle  rider 
could  sustain.   Therefore,  NHTSA  conducted 
another  analysis  that  confirmed  that 
motorcycle  helmets  were  67  percent 
effective  at  preventing  brain  injuries. 

None  of  these  analyses  would  have 
been  possible  without  data  on  EMS 
transport,  emergency  department  treatment, 
or  inpatient  treatment.   The  information 
for  an  accurate  assessment  of  the  injury 
severity  which  was  necessary  for  the 
different  effectiveness  evaluations  was  not 
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available  on  the  crash  file.   The  analysis 
of  motorcycle  helmet  effectiveness  for 
preventing  brain  injury  could  not  have  been 
conducted  without  being  able  to  identify 
the  type  of  injury  sustained  by  inpatient 
motorcycle  riders.   Linking  files  obtained 
these  data. 

Cost  of  treatment  is  extremely 
difficult  to  obtain.   The  cost  data 
available  in  the  CODES  project  were 
inpatient  charges.   NHTSA  statistically 
combined  average  inpatient  charges  from 
each  of  the  CODES  states  to  produce  overall 
average  charges  for  belted  and  unbelted 
occupants  and  helmeted  and  helmeted 
drivers.   In  both  cases,  the  average 
inpatient  charges  for  those  drivers  or 
riders  electing  not  to  use  the  protection 
equipment  were  significantly  higher.   These 
results  are  shown  in  Exhibit  3.   Without 
being  able  to  link  to  the  hospital 
discharge  or  inpatient  files,  data  on 
charges  would  not  have  been  available. 


Exhibit  3. 

Average  Inpatient  Charge  by  Protection 

Equipment  Use 

for  Inpatient  Passenger  Vehicle 

Drivers  and  Motorcycle  Riders , 

in  the  CODES  States 

Group 

Protection 
Equipment 

Increase 

for 

Nonuse 

Used 

Not 
Used 

Passenger 

Vehicle 

Drivers 

$8,  174 

$13, 144 

61% 

Motorcycle 
Riders 

$12, 374 

$15, 447 

2  5% 

information  NHTSA  is  encouraging  states  to 
develop  their  own  CODES.   We  believe  that 
injury  control  in  highway  safety  needs  data 
such  as  that  which  can  be  provided  from  a 
CODES . 

Many  states  are  not  able  to  develop  a 
CODES,  however.   They  lack  one  or  more  of 
the  necessary  medical  outcome  data  files  -- 
EMS,  emergency  department,  hospital 
discharge.   Three  states  even  lack  a 
statewide,  computerized  crash  database. 
So,  many  states  need  to  be  encouraged  to 
create  these  databases.   In  the  meantime 
NHTSA  will  be  promoting  the  advantages  of 
these  databases,  their  linkage,  and  their 
use.   Information  about  CODES  and  results 
from  the  analyses  performed  by  the  CODES 
states  will  be  presented  at  conferences  and 
workshops.   We  will  be  making  expertise 
gained  by  the  CODES  states  available  to 
those  states  who  need  it  to  get  started.   A 
contract  is  being  developed  to  allow  new 
states  to  request  this  input. 

Obviously,  there  is  a  long  way  to  go 
before  a  significant  number  of  states  have 
this  resource  available  to  them.   We 
believe  it  is  a  necessary  tool  for  future 
highway  safety  analyses. 


Data  linkage  has  obvious  advantages . 
The  data  in  the  police  crash  database  is 
enhanced.   When  linked,  these  data  can  be 
used  for  many  more  analyses  than  they  could 
when  the  medical  outcome  data  were  not 
associated  with  the  involved  persons. 

But  there  were  other  advantages  which 
were  not  anticipated  at  the  onset  of  the 
CODES  project.   To  accomplish  linkage,  data 
files  must  be  scrutinized  in  detail.   This 
led  to  standardization  of  similar  data 
items  on  the  various  files.   Additionally, 
this  detailed  review  of  the  data  was  often 
the  first  exhaustive  look  at  some  of  the 
databases.   This  led  to  many  quality 
improvements  in  the  data.   In  one  state, 
the  data  processing  contractor  was  replaced 
when  the  poor  job  being  done  was 
discovered.   Finally,  each  state  convened 
an  advisory  committee  of  owners  and  users 
of  the  data.   Often,  this  was  the  first 
time  many  of  these  people  had  collaborated 
on  projects. 

What's  next  in  data  linkage?   NHTSA' s 
vision  of  injury  control  for  highway  safety 
needs  environmental,  cause  of  injury,  and 
medical  outcome  information  that  is  useful 
to  identify  problems,  develop  programs  and 
set  and  evaluate  priorities.   To  get  this 
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ABSTRACT 

This  paper  presents  selected  findings  from  a 
1994  telephone  survey  of  policymakers  in  all  50 
states  funded  by  the  Robert  Wood  Johnson  Foundation 
and  fielded  in  January- -March  1994.  The  focus  is  on 
findings  relevant  to  understanding  where  we  start 
in  having  adequate  data  to  monitor  access  to  health 
care  in  each  state.  Doing  so  is  likely  to  become 
increasingly  important  over  time  as  policy 
authority  gets  decentralized  to  the  states  and 
interest  grows  in  understanding  how  changes  in  both 
the  public  and  private  health  systems,  such  as 
funding  cutbacks  and  managed  care,  affect  access  to 
health  care  services  across  the  nation. 

In  sum,  the  results  indicate  that  states  vary- 
substantially  in  the  information  they  have  to 
monitor  access  but  also  that  there  are  serious 
limitations  in  the  access  measures  available  in 
each  of  the  states.  For  example,  state  policymakers 
cannot  easily  tell  the  number  and  characteristics 
of  those  without  coverage  in  the  state  since  most 
rely  on  national  surveys  which  generally  were  not 
developed  for  this  purpose.  Virtually  none  of  the 
states  has  data  on  how  satisfied  consumers  are  with 
health  plans  or  with  the  quality  of  care  in  these 
plans.  At  a  time  when  changes  in  the  role  of  public 
health  departments  in  delivering  primary  care  are 
being  considered,  the  majority  or  more  of  state 
health  department  respondents  do  not  have  very  much 
confidence  in  data  showing  an  unduplicated  count  of 
clinic  users  or  the  alternative  sources  of  care 
available  to  clinic  users. 

Identifying  which  of  these  gaps  are  most 
important  and  how  to  address  them  should  be  a 
priority  health  policy  issue  at  the  national  level, 
both  to  allow  monitoring  against  national  health 
access  objectives  and  also  to  support  states  in 
carrying  out  their  responsibilities. 

SUMMARY  OF  PRESENTATION 

Rationale  for  This  Study 

Current  trends  in  the  health  care  system  are 
likely  to  make  the  monitoring  of  changes  in  health 
care  particularly  important  over  time,  especially 
at  the  state  level.  States  play  an  increasingly 
important  role  in  addressing  a  complex  array  of 
health  policy  issues  and  challenges.  These  include 
tensions  created  by  efforts  to  use  the  Medicaid 
program  to  promote  access  in  an  era  of  escalating 
costs;  the  resurgence  of  traditional  public  health 
issues  associated  with  infectious  disease  and 
poverty;  long-standing  concerns  with  prevention, 
health  promotion,  and  the  development  and 
distribution  of  health  care  personnel  facilities 
and  services;  and  newer  concerns  relating  to 
integrating  public  and  private  efforts  to  handle 
these  issues. 

Appropriate  tools,  including  data  and 
analytical  capability,  are  an  important  part  of  the 
infrastructure  through  which  policymakers  can 
address  these  challenges.  Yet,  the  quality  and 
sophistication  of  these  tools  vary  considerably 
from  state  to  state,  and  critical  weaknesses  are 
apparent  in  even  the  most  advanced  states.  In 
addition  to  limiting  what  states  can  accomplish, 
these  shortcomings  lessen  the  ability  nationally  to 
monitor  changes  such  as  in  access  that  vary  across 
states  in  response  to  differences  in  both  health 
care  market  trends  and  state  policy. 

This  paper  aims  to  (1)  assess  both 
policymakers'  confidence  in  data  relevant  to  access 
and  the  characteristics  of  available  data;  (2) 
assess  funding  trends  and  barriers  to  improving  the 


utility  of  data;  and  (3)  identify  the  key  policy 
issues  related  to  enhancing  data  for  monitoring 
access . 

Survey  Design  and  Methods 

The  results  in  this  paper  are  based  on  a 
telephone  survey  of  key  senior  state  officials  in 
the  legislative  and  executive  branches  of  all  50 
states  and  the  District  of  Columbia.'  The 
interviews  were  conducted  between  January  and  March 
1994  by  executive  interviewers  in  the  Survey  Center 
at  Mathematica  Policy  Research. 

Interviewers  were  conducted  by  telephone  with 
senior  health  policy  advisors  in  eight  functional 
areas:  governor's  health  aide,  health  analysis  for 
legislative  committees  (up  to  two  in  each  state) ; 
central  budget  department  staff  dealing  with  health 
programs;  a  health  reform  entity,  that  is,  the  lead 
agency  or  task  force  charged  with  responsibility 
for  health  reform;  Medicaid  agency  policy  analysis ,- 
public  health  agency  policy  analysis;  vital  and 
population-based  health  statistics;  and  the 
database  commission  or  other  agency  responsible  for 
health  resource  and  utilization  analysis.  Responses 
were  received  from  442  of  452  potential 
respondents,  with  at  least  seven  responses  for  each 
state . 

Survey  content  varied  by  respondent.  All  were 
asked  a  core  set  of  items  on  their  confidence  with 
data  relevant  to  their  concerns  and  other  issues. 
The  first  three  (governor's  aides,  legislative 
aides,  and  budget  staffers) ,  termed  "central 
policymakers,"  were  asked  about  their  perspectives 
across  a  range  of  issues.  The  last  five  were  asked 
about  their  areas  of  concern  and  were  also  asked  to 
provide  factual  information  about  data  the  state 
actually  collects  in  their  area  of  responsibility. 
In  discussing  the  survey  findings  presented  here, 
we  indicate  for  each  set  whom  the  relevant 
respondents  were. 

FINDINGS 

Central  Policymaker  Confidence  in  the  Ability 
of  Health  Data  to  Address  Priority  Health  Issues. 
We  asked  central  policymakers  to  rank  12  issues  in 
terms  of  priority  in  early  1994  (Figure  1) .  Of  the 
12,  five  were  clearly  dominant  in  early  1994: 
Medicaid,  cost  containment,  access  to  care  and  the 
uninsured,  maternal  and  child  health,  and  managed 

care.  State  officials  were  most  confident  in  data 
on  Medicaid,  followed  by  maternal  and  child  health. 
Less  than  10  percent  were  very  confident  about  data 
available  on  the  other  three  issues— all  of  which, 
in  different  ways,  are  important  in  examining 
access  (Figure  2) . 

Data  on  Insurance  Coverage,  Expenditures  and 
Health   System/Plan   Performance   in   the   State. 

Governor's  aides,  legislative  aides,  and  health 
reform  staffers  were  asked  about  their  confidence 
with  state-based  data  on:  health  insurance 
coverage,  health  expenditures,  and  health 
system/health  plan  performance.  These  were  viewed 
as  "very  important"  by  87%,  98%,  and  73%  of  state 
respondents . 

Overall,  few  respondents  (less  than  10%) 
perceived  any  of  the  insurance  coverage  data  to  be 
excellent  and  over  half  perceived  each  type  to  be 
no  better  than  fair  (Figure  3) .  Aggregate  data  on 
the  total  percentage  uninsured  in  the  state 
generated  more  confidence  than  any  of  the  other 
kinds  of  data.  Factual  reports  by  health  reform 
staffers  show  that  most  states  appear  to  rely  on 
national  data  sources  for  information  on  insurance 
coverage  even  though  only  one  source  at  best  (the 
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Current  Population  Survey)  is  designed  for  this 
purpose.  The  quality  of  state-collected  data 
appears  very  uneven. 

Confidence  in  health  expenditure  data  was  much 
greater  for  services  provided  in  hospitals  or  by 
state  government  than  overall  (Figure  4) .  Virtually 
none  of  the  states  good  confidence  in  data  on  total 
consumer  out  of  pocket  spending.  These  results  are 
consistent  with  the  structure  of  data  collection 
currently  in  each  state.  Reports  by  health 
resource /database  staffers  show,  for  example,  that 
reporting  is  much  more  extensive  for  hospitals  and 
other  facilities  than  for  community  based  providers 
and  practitioners,  few  of  whom  report  any  but 
perhaps  the  most  basic  provider  characteristic 
information  (Table  1) . 

Confidence  with  data  on  different  measures  of 
health  system  and  plan  performance  was  lowest  of 
the  three  areas  examined,  with  few  states  having 
better  than  fair  confidence  in  data  on  quality  of 
care  for  plans  or  providers  or  consumer 
satisfaction  with  care  or  insurance  (Figure  5) . 

Data  on  Public  Health  Services.  Figure  6  shows 
the  relative  confidence  of  public  health 
respondents  in  each  state  about  specific  kinds  of 
data  we  perceived  might  help  them  decide  if  funding 
for  public  health  clinics  should  be  reduced  as  part 
of  state  budget  funding.  Confidence  was  most  on  the 
ability  to  identify  the  number  of  clinic  visits  in 
the  past  year.  It  was  least  on  the  ability  to 
identify  alternative  sources  of  care  for  clinic 
users  and  on  the  effects  of  the  clinics  on  health 
outcomes.  An  inability  to  obtain  and  especially  to 
link  information  on  individual  services  provided  to 
different  patients,  in  different  clinics,  or  in 
different  jurisdictions  probably  is  one  reason  such 
information  is  hard  to  obtain  (Figure  7) . 

Barriers  to  Improving  the  Availability  and 
Quality  of  Data.  The  three  biggest  barriers 
Medicaid,  public  health,  resource/database  and 
vital  statistics  respondents  view  in  improving  data 
are  lack  of  comparability  across  datasets,  funding, 
and  gaining  submission  of  data  from  providers  and 
insurers  (Figure  8)  . 

Trends  and  Fiscal  Issues.  Sixty-eight  percent 
of  all  respondents  surveyed  perceived  that  the 
ability  of  health  data  to  meet  policy  needs  had 
improved  from  three  years  ago,  while  7  percent  felt 
it  was  worse,  and  25  percent  saw  no  change.  At 
least  as  of  early  1994,  data  did  not  appear  to  be 
losing  funding  relative  to  general  cutbacks,  but 


the  cuts  have  had  some  negative  effects:  84  percent 
of  states  reported  more  pressure  on  staff,  and  34 
and  26  percent  reported  a  reduced  scope  of  analysis 
or  timeliness  of  reporting. 

CONCLUSIONS  AND  POLICY  ISSUES 

State  data  systems  and  the  confidence 
policymakers  have  in  them  reflect  the  history  of 
state  health  programs  and  sources  of  support. 
States  appear  relatively  confident  in  data  that 
supports  ongoing  operations  of  stable  programs  such 
as  monitoring  Medicaid  or  operating  individual 
public  health  programs.  However,  state  data  systems 
are  not  perceived  as  being  well -suited  to 
supporting  assessments  of  program  needs  or  to 
guiding  decisions  about  restructuring  health  care 
systems  in  a  changing  environment. 

The  findings  suggest  that  serious  gaps  exist  in 
data  potentially  of  great  value  in  monitoring 
access  across  or  within  states.  Particularly  in 
today's  fiscal  climate,  it  is  important  to  identify 
priorities.  What  data  are  needed  across  all  states? 
These  clearly  have  national  priority,  the  key  issue 
being  to  decide  which  need  to  be  collected 
centrally  and  which  by  each  state.  How  to  structure 
and  finance  such  data  initiatives  also  are  key, 
particularly  when  budgets  are  being  cut  back  and 
the  desirable  data  probably  are  substantially  in 
excess  of  that  which  can  be  financed.  Nationally, 
there  could  be  a  need  to  consider  trade-offs 
between  depth  and  breadth.  Having  comprehensive 
national  data  systems  has  contributed  to  enhanced 
understanding  of  access  in  the  U.S.  Perhaps,  at  the 
margin,  some  depth  could  be  sacrificed  to  support 
larger  sample  sizes  that  can  better  support  state - 
specific  and  timely  estimates  of  key  measures. 
Given  the  changing  policy  context,  having  such 
information  could  be  of  growing  importance . 
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Table  1 .  Characteristics  of  Data  States  Collect  From  Providers 

Number  of  States  That  Collect  by  Data  Type 

Provider 

Pt 

Level 

Acute  Care 
Hospitals 

%  Any 

(N) 

Any 

Charac- 
teristics 

Aggregate 
Use 

Finance 

Discharge/ 
Encounter 

92 

(51) 

47 

33 

40 

36 

38 

Ambulatory  Care 
Facilities 

51 

(51) 

26 

19 

18 

16 

16 

Licensed  Clinics 

30 

(50) 

15 

11 

7 

7 

8 

Physician 

53 

(51) 

27 

25 

5 

1 

6 

Nurse  Midwives 

34 

(50) 

17 

14 

3 

1 

4 

Physician  Assistant 

37 

(51) 

19 

16 

1 

0 

2 

SOURCE:  RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
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Figure  1.  Health  Issues  Ranked  by  Priority 


SOURCE:    RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
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Figure  2.  Policymaker  Confidence  in 

Health  Data  to  Address  Priority  Issues 
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SOURCE:    RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
NOTE:         Other  categories  were  not  very  or  not  at  all  confident. 
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Figure  3.  Perceived  Quality  of  Different  Types  of  Insurance  Coverage  Data 


No.  of  People  with  No  Coverage 


By  Age,  Sex,  and  Income 


By  Employment  Status  and 
Occupation 

By  Geographic  Subdivision 


Type  of  Services  Covered  and 
Required  Cost  Sharing 


Employers  Not  Offering 
Health  Insurance 


Amount  of  Employers'  Contribution 
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SOURCE:    RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
NOTE:         NA  =  Not  available. 

Figure  4.  Perceived  Quality  of  Different  Types  of  Health  Expenditure  Data 


Total  Health  Care 


Total  Hospital 
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Total  State  Government 
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SOURCE:    RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
NOTE:  NA  =  Not  available. 
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Figure  5.  Perceived  Quality  of  Different  Types  of  Health  System/Plan  Performance  Data 
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SOURCE:    RWJF  1994  State  Survey  by  Mathematics  Policy  Research. 
NOTE:         NA  =  Not  available. 


Figure  6.  Confidence  in  Health  Data  to  Provide  Specified  Public  Health  Information,  Public  Health  Respondents 
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SOURCE:  RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
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Figure  7.  Reported  Characteristics  of  State  Public  Health  and  Data  Reporting 
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SOURCE:    RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
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Figure  8.  Barriers  to  Improving  the  Availability  or  Quality  of  Data  for  Health  Reform 
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SOURCE:  RWJF  1994  State  Survey  by  Mathematica  Policy  Research. 
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MONITORING  ACCESS  IN  STATES  WITH  MEDICAID  WAIVERS 
ROLE  OF  THE  BUREAU  OF  PRIMARY  HEALTH  CARE 

Richard  C.  Lee,  Bonnie  Lefkowitz 
Bureau  of  Primary  Health  Care 


Introduction 

Many  States  are  setting  up  Health 
Care  Reform  plans  through  the  medium 
of  Medicaid  waiver  demonstration 
projects.   The  declared  objectives  of 
these  projects  are  to  improve  access 
for  Medicaid-eligibles  and  uninsured 
persons  while  simultaneously  reducing 
the  costs  of  providing  this  care.   The 
Health  Care  Financing  Administration 
(HCFA) ,  because  of  its  responsibility 
for  the  Medicaid  program,  is 
responsible  for  approving  and 
monitoring  these  waiver  projects. 

Medically  underserved  populations 
in  every  State,  including  both 
Medicaid  recipients  and  other  low- 
income  uninsured  persons,  are 
currently  being  served  through  a 
variety  of  programs  funded  by  the 
Bureau  of  Primary  Health  Care  (BPHC) 
under  the  Public  Health  Service  Act. 
These  vulnerable  populations  are  also 
served  by  other  Federally  Qualified 
Health  Centers  (FQHCs) ,  typically  run 
by  State  or  local  health  departments 
or  community  groups,  which  have  been 
identified  by  BPHC  as  providers  of 
primary  care  services  to  underserved 
persons  and  which  "look  like"  (i.e. 
meet  the  same  requirements  as)  the 
BPHC- funded  programs.   (The  term  FQHCs 
is  defined  to  include  both  BPHC-funded 
projects  and  "look-alikes".) 

By  using  FQHCs,  many  previously 
unserved  or  underserved  patients  have 
achieved  access  to  continuous,  quality 
primary  health  care  at  relatively  low 
cost  to  taxpayers.   Therefore,  BPHC 
and  HCFA  have  mutual  concerns  that  the 
State  Medicaid  waiver  plans  are 
implemented  in  such  a  way  as  to  assure 
continued  access  to  quality  health 
care  for  these  patients. 

One  way  that  these  concerns  have 
been  addressed  has  been  by  HCFA' s 
including,  in  the  "Terms  and 
Conditions"  of  many  of  the  waivers  it 
has  issued,  a  requiremenc  that  "the 
State  shall  require  managed  care 
organizations  (MCOs)  to  contract  with 
FQHCs  unless  the  MCO  demonstrates  that 
both  adequate  capacity  and  appropriate 
facilities  for  vulnerable  populations 
exist  in  its  area  without  the  use  of 
FQHCs."   The  Terms  and  Conditions  have 
also  frequently  included  Access 
Standards,  including  a  requirement 
that  the  location  where  a  patient  will 
receive  services  should  be  within  3  0 
miles  or  30  minutes  of  the  patient's 
residence.   In  addition,  most  of  the 


Terms  and  Conditions  require  the  State 
to  monitor  the  number  and  types  of 
providers  before  and  after  the  waiver, 
and  various  measures  of  recipient 
access  to  services,  including  waiting 
times . 

However,  data  are  needed  in  order 
to  make  an  informed  review  of  waiver 
proposals  and  of  implementation  plans 
addressing  the  general  issue  of 
assuring  access  to  vulnerable 
populations  and/or  the  more  specific 
FQHC  issue,  as  well  as  to  monitor  the 
plans  when  implemented.   These  may 
include  data  on  the  location  of 
vulnerable  populations,  data  on 
indicators  of  their  access  to  primary 
health  care,  and  data  on  the  location 
of  existing  FQHCs.   BPHC  has  an 
extensive  data  base  on  variables 
relevant  to  these  issues,  and  is 
making  such  data  available  to  HCFA  for 
use  in  both  analyzing  waiver  requests 
and  monitoring  access  in  States  where 
waiver  plans  have  been  implemented. 

BPHC  Programs  for  Vulnerable 
Populations 

BPHC  is  responsible  for  six  major 
programs  targeted  at  underserved  areas 
and  populations.   They  are: 

(1)  Community  Health  Centers  (CHCs) ; 

(2)  Migrant  Health  Centers; 

(3)  Public  Housing  Clinics; 

(4)  the  National  Health  Service  Corps; 

(5)  Health  Care  for  the  Homeless;  and 

(6)  Ryan  White  Title  3B  HIV  projects. 
These  programs  involve  2500  total 
sites  providing  9  million  people  with 
comprehensive  preventive  and  primary 
care,  case  management  of  specialty  and 
inpatient  services,  and  various 
enabling  and  facilitating  services. 

These  centers/sites  have  the 
potential  of  making  a  major 
contribution  to  Medicaid  managed  care. 
They  already  serve  4  million  Medicaid 
beneficiaries,  together  with  many 
uninsured  persons.   They  are  the  only 
source  of  care  in  many  areas .   Many 
already  participate  in  managed  care 
through  Integrated  Service  Networks 
supported  or  encouraged  by  BPHC.   As 
of  December  31,  1994,  157  CHCs  were 
prepaid  plan  providers,  with  a  total 
enrollment  of  566,076.   Studies1-2,3 
indicate  that  these  projects  provide 
effective  and  efficient  primary  care, 
and  that  their  services  to  Medicaid- 
eligible  users  are  less  costly  when 
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compared  with  care  obtained  by 
Medicaid-eligibles  in  the  same  service 
areas  who  are  non-users  (or  only 
occasional  users)  of  CHCs." 

BPHC  Data  Bases  and  Data  Collection 
Activities 

BPHC  has  a  variety  of  databases 
which  contain  information  relevant  to 
assessing  and  monitoring  access  of 
vulnerable  populations  to  primary 
health  care.   These  include  both 
databases  on  the  projects  it  funds  and 
databases  relevant  to  assessment  of 
areas'  and  population  groups'  needs 
for  and  access  to  primary  medical 
care.   There  are  also  data  collection 
activities  now  in  progress  which  will 
allow  analysis  on  a  sample  of 
projects.  These  databases  and  data 
collection  activities  include: 

(1)  The  Bureau  Common  Reporting 
Requirement  (BCRR)  database, 
maintained  by  BPHC  and  its  predecessor 
Bureaus  since  the  1970s.   This 
database  contains,  for  each  project  or 
connected  group  of  projects, 
information  on  the  staff  and  resources 
available,  volume  of  services 
provided,  and  number  of  users. 

BPHC's  new  Uniform  Data  System 
(UDS)  reporting  system,  effective 
January  1996,  will  add  demographics  of 
the  user  population,  crosswalk  to  the 
HCFA  Medicare/Medicaid  reporting 
system,  and  data  on  selected 
diagnoses . 

(2)  BPHC  has  worked  with  NCHS  to 
design  and  conduct  user/visit  surveys 
of  health  centers  on  the  Health 
Interview  Survey  (HIS)  and  NHAMCES 
models,  in  order  to  study  access, 
diagnoses,  services,  continuity, 
satisfaction,  and  outcomes  in  the  CHC 
environment.   The  results  of  those 
surveys  are  forthcoming.   A  current 
project  will  adapt  these  user/visit 
surveys  for  waiver  States. 

(3)  BPHC  assists  each  grantee  with 
clinical  self -evaluation  measures 
corresponding  to  HEDIS  quality 
standards . 

(4)  The  Health  Professional  Shortage 
Area  (HPSA)  database.   HPSAs  are 
areas,  population  groups  and 
facilities  which  have  been  determined 
to  have  shortages  of  health 
professionals,  and  are  therefore 
eligible  to  apply  for  placement  of 
National  Health  Service  Corps  (NHSC) 
personnel .   The  HPSA  database  contains 
data  on  every  county  and  every 
designated  HPSA  in  the  United  States, 
including  the  definition  of  each 
service  area  in  terms  of  counties, 
census  tracts  or  census  divisions; 
population  of  each  area  or  population 
group;  number  of  full -time -equivalent 

(FTE)  primary  care  physicians  serving 
the  HPSA;  ratio  of  population  to  FTE 


primary  care  physicians,  poverty  rate, 
and  other  variables.   For  all 
designated  HPSAs,  the  database  also 
contains  the  estimated  travel  time  and 
distance  to  the  nearest  source  of  care 
outside  the  HPSA.   County-level  data 
on  infant  mortality  rate  and  rate  of 
low  birthweight  births  are  also 
included.   Each  HPSA  is  required  to  be 
updated  every  three  years  to  remain 
designated. 

(5)  The  Medically  Underserved 
Area/Population  (MUA/P)  database. 
MUAs  are  areas  which  have  been 
designated  as  having  a  shortage  of 
primary  care  services  based  on  their 
scores  on  the  four-variable  Index  of 
Medical  Underservice  (IMU) ;  MUPs  are 
population  groups  designated  as  having 
such  a  shortage  either  based  on  an  IMU 
score  or  a  Governor's  recommendation. 
Areas  and  populations  designated  as 
MUA/Ps  are  eligible  to  apply  for  CHC 
funding.   This  database  identifies 
each  designated  MUA  or  MUP  and  its  IMU 
score  or  status  as  a  Governor's 
request . 

BPHC  is  currently  developing  a 
new  method  of  designation  to  replace 
both  the  HPSA  and  MUA/P  systems  with 
one  self -consistent  system.   It  would 
include  many  of  the  same  variables  now 
used  in  HPSA  and/or  MUA  designation, 
and  others  as  well. 

(6)  The  BPHC  "Geofile",  containing 
census  demographic  data  for  all 
counties,  census  tracts  and  census 
divisions;  AMA/AOA  physician  data  by 
county;  NCHS  mortality  and  natality 
data  by  county;  and  other  variables. 

(7)  BPHC  has,  through  Cooperative 
Agreements  with  all  the  States, 
stimulated  the  development  and 
submission  of  State  Primary  Care 
Access  Plans,  which  contain 
information  on  the  number  of  primary 
care  providers  and  the  unmet  need  for 
primary  care  providers,  by  county  (and 
for  subcounty  service  areas  if  defined 
by  the  State) .   We  are  currently 
assembling  a  database  from  this  data, 
and  plan  to  link  it  to  the  other 
databases  identified  above. 

Schematic  for  Access  Indicators 

Below  is  a  schematic  of  BPHC's 
anticipated  efforts  in  monitoring 
access  in  waiver  States. 

In  the  baseline,  we  examine  the 
gaps  in  provider  capacity  within  the 
areas  the  State  has  agreed  (through 
the  designation  process)  are 
underserved.   This  would  include 
examining  the  extent  to  which  these 
gaps  are  currently  being  filled 
through  Federal  interventions  such  as 
NHSC  providers,  CHCs,  or  other  FQHCs . 

Through  the  Primary  Care  Access 
Plan  process  and  through  the  review  of 
implementation  plans  submitted  in  the 
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context  of  waiver  requests,  the  States 
are  queried  as  to  how  they  intend  to 
assure  adequate  capacity,  and  what 
role  Federal  interventions  will  play. 

In  the  coming  months,  BPHC  will 
assemble  baseline  data  from  the 
HPSA/Geofile  databases  for  the  12  or 
more  currently- approved  waiver  States, 
as  well  as  for  those  with  waiver 
applications  under  review  by  HCFA. 
To  this  will  be  added  data  on  CHCs  and 
FQHCs  from  the  BCRR  database  and  other 
sources,  and  data  from  the  Primary 
Care  Access  Plans. 

This  baseline  information  is 
available  for  structure  and  process 
elements  currently. 


SCHEMATIC  FOR  ACCESS  INDICATORS 
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Next  Steps 

In  the  future,  BPHC  will  work  to 
deepen  the  baseline  information, 
adding  elements  on  intermediate 
outcomes  and  health  status  from  the 
User/Visit  Survey  and  other  evaluation 
projects . 

In  addition,  updates  will  also  be 
scheduled,  and  compared  with  the 
baseline,  first  on  a  provider-specific 
basis  and  then  also  on  an  areawide 
basis  as  new  information  becomes 
available  from  the  Primary  Care  Access 
Plan  and  designation  databases. 

Updates  of  outcome  and  health 
status  information  will  similarly  be 
available  first  for  BPHC-funded 
providers  and  then,  as  surveys  are 
adapted  for  wider  use,  for  areas  as 
well . 


References 

1Starfield,  Barbara  et  al . ,  "Costs  vs. 
Quality  in  Different  Types  of  Primary 
Care  Settings."   JAMA,  Vol.  278, 
No.  24  (Dec.  28,  1994) :  1903-1908 

2Stuart,  Mary  and  Steinwachs,  Donald, 
"Patient-Mix  Differentials  Among 
Ambulatory  Providers  and  Their  Effects 
on  Utilization  and  Payments  for 
Maryland  Medicaid  Users."   Medical 
Care,  Vol.  31,  No.  12:  1119-1137. 

3Lewin-VHI,  Inc.  and  MDS  Associates, 
"Community  Health  Centers'  Performance 
Under  Managed  Care:  Executive 
Summary."   PHS-HRSA-BPHC  Report, 
December  1994. 

4Rodgers,  Judy,  Duggar,  Ben,  et  al .  , 
"Utilization  and  Cost  of  Medicaid 
Services  Provided  to  Users  and  Non- 
Users  of  Community  Health  Centers," 
publication  forthcoming. 


289 


MONITORING  ACCESS  TO  HEALTH  CARE  AT  THE  COMMUNITY  LEVEL 


Jeannette  Jackson-Thompson,  Missouri  Department  of  Health 
Bernard  Ewigman  and  Robin  Kruse 


Introduction 


Access  to  health  care  can  be  defined  as  the  ability  to  obtain 
appropriate  health  care  when  it  is  needed.  Access  "implies  that 
people  have  a  place  to  go  and  the  financial  and  other  means  of 
obtaining  care."  Many  people  in  this  country  do  not  have 

a  place  to  go  and  are  unable  to  obtain  health  services  they  need, 
either  because  of  the  high  cost  of  needed  services  or  the  lack  of 
service  availability  in  their  geographic  area.  People  without 
easy  access  may  delay  seeking  care;  they  may  face  barriers  that 
delay  access  to  care;  and  they  may  use  the  emergency  room 
(ER)  as  a  primary  care  substitute.  Long  delays  in  seeking 
and/or  accessing  primary  care  and  inappropriate  ER  use  lead  to 
lowered  health  status  for  individuals  and  place  an  increased 
burden  on  some  providers. 

The  Missouri  Department  of  Health  (MDOH)  has  established 
as  one  of  eight  strategic  goals  for  the  year  2000  a  goal  of 
universal  access: 

By  2000  quality  preventive,  primary,  acute  and 
rehabilitative  care  will  be  universally  available  and 
accessible  to  all  Missourians.     :) 

MDOH  is  especially  concerned  with  lack  of  access  to 
preventive  and  primary  care,  the  effective  use  of  which  should 
ease  some  of  the  personal  and  financial  burdens  of  acute  care. 
To  measure  access  at  the  state  level,  MDOH  added  a  four- 
question,  state-developed  module  on  access  to  care  to  its  1990 
Behavioral  Risk  Factor  Surveillance  System  (BRFSS) 
questionnaire.  The  same  module  was  also  used  in  two  special 
local  area  surveys  conducted  by  MDOH  in  1990.  Regional 
differences  were  observed  in  access.  For  example,  9.6%  of 
BRFSS  respondents  statewide  reported  they  had  no  form  of 
health  insurance,  compared  with  13.5%  of  respondents  in  one 
six-county  area  and  16.0%  of  respondents  in  eight  inner-city 
neighborhoods. 

To  plan  effectively  for  adequate  health  services  so  that  timely 
and  appropriate  care  is  available  to  all  who  need  it,  local  level 
information  on  utilization  patterns,  access  to  health  care  and 
barriers  to  care  is  needed.  Such  information  is  frequently 
lacking. 

In  1992,  health  care  planners  and  providers  had  no  clear 
understanding  of  how  many  residents  of  Boone  County, 
Missouri,  had  problems  getting  needed  health  care  or  the 
barriers  which  prevented  residents  from  accessing  the  health 
care  system.  Despite  an  apparent  abundance  of  physicians  and 
hospital  beds  in  the  county,  providers  in  the  community 
believed  that  many  Boone  County  residents  were  unable  to 
access  and  receive  appropriate  health  care  in  a  timely  fashion. 
Their  belief  led  to  the  establishment  of  a  new  clinic,  the  Boone 
County  Family  Health  Center  (now  the  Family  Health  Center  at 
Parkade),  which  serves  uninsured  and  underinsured  Boone 
County  families  and  individuals.  The  lack  of  knowledge  about 


access  and  barriers  plus  the  presence  of  the  clinic  provided  the 
impetus  for  the  research  reported  here,  a  population-based 
survey  of  Boone  County  adults. 

This  project,  a  collaborative  effort  involving  both  public  and 
private  partners,  was  undertaken  to  assess  current  health  needs 
in  the  community,  particularly  of  low-income  and  underinsured 
residents,  and  provide  a  baseline  for  measuring  progress  toward 
the  goal  of  universal  access  to  basic  health  care.  MDOH 
conducted  random-digit-dialed  (RDD)  telephone  interviews 
with  897  adult  (>  18  years  of  age)  residents  of  Boone  County, 
Missouri  during  October  1992.  The  study  was  funded  by 
Preferred  Health  Plans  of  Missouri.  The  survey  instrument, 
developed  by  MDOH,  the  Family  Health  Center  at  Parkade  and 
the  City  of  Columbia/Boone  County  Health  Department, 
included  questions  from  the  1992  Missouri  BRFSS 
questionnaire  plus  additional  questions  on  access,  utilization, 
need  and  perceived  banners  to  care.  Results  have  been  used  to 
determine  the  prevalence  of  specific  barriers  to  health  care 
access  and  the  populations  most  affected  by  these  barriers.  The 
survey  is  also  being  used  to  develop  a  model  for  continued 
measurement  of  health  care  access,  utilization  and  need  as 
managed  care  becomes  more  prevalent  in  Missouri. 

Methods 

The  specific  aims  of  the  study  were  1)  to  determine  the 
prevalence  of  adult  county  residents  experiencing  barriers  to 
health  care  by  type  of  barrier  and  2)  to  characterize  the 
populations  experiencing  barriers.  The  study  was  based  on 
BRFSS  methods  and  techniques.11'12  A  simple  random 
sampling  frame  was  used  rather  than  the  modified  Waksberg 
cluster  sampling  used  for  Missouri  BRFSS  data  collection. 
Starting  with  a  list  of  all  residential  telephone  prefixes  known  to 
be  in  use  in  Boone  County,  MDOH  staff  generated  10,000 
random  telephone  numbers.  Two  prefixes  assigned  to  the 
University  of  Missouri-Columbia  (UMC)  were  not  included 
among  residential  prefixes,  thus  excluding  all  UMC  students 
living  in  university  housing.  These  students  had  access  to  an 
on-site  student  health  clinic  and  could  be  referred  to  UMC 
Hospital  and  Clinics  through  a  mandatory  campus  health  care 
plan.  Full-time  students  living  off-campus  were  included  in  the 
survey  sample  but  were  excluded  from  most  analyses  for  the 
same  reason. 

Data  Collection.  Interviews  were  conducted  during  October 
1992  by  trained  interviewers.  A  two-stage,  RDD  technique  was 
used.  During  the  first  stage,  most  nonworking  and  ineligible 
telephone  numbers  were  eliminated.  In  the  second  stage,  a 
respondent  was  identified  from  among  all  adult  residents  of  a 
household.  A  state-developed,  menu-driven  CATI  (computer- 
assisted  telephone  interviewing)  system  loaded  on  stand-alone 
personal  computers  (PCs)  was  used  for  data  collection.  Calls 
were  made  weekdays,  evenings  and  weekends.  Up  to  twenty 
attempts  (four  attempts  over  five  different  calling  periods)  were 
made  to  reach  each  RDD  number.  Only  one  respondent  in 
each  household  was  eligible  for  inclusion  in  the  study. 
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Successful  contacts  that  resulted  in  897  completed  interviews 
were  made  with  1,055  eligible  households  (defined  as  a  private 
residence  in  Boone  County  with  at  least  one  person  18  years  of 
age  or  older  living  in  the  household),  for  a  completion  rate  of 
85%.  Of  the  remaining  158  residential  numbers,  133  potential 
respondents  refused  to  participate.  Another  25  respondents 
were  unavailable  during  the  survey  period,  started  but  did  not 
complete  an  interview  or  were  ineligible  for  inclusion  in  the 
study  due  to  age  (<  18)  or  selection  procedure  (wrong 
household  member  selected). 

Survey  Instrument.  The  survey  instrument  contained  77 
questions  and  was  a  modified  version  of  the  1992  Missouri 
BRFSS  questionnaire.  Twenty-six  percent  of  the  questions 
were  developed  or  adapted  from  non-BRFSS  sources  for  this 
survey;  64%  were  core  BRFSS  questions  and  10%  were  state- 
added  BRFSS  questions  already  in  use.  MDOH  and  University 
of  Missouri-Columbia  faculty  and  staff  analyzed  data. 

Key  Questions  on  Barriers  to  Health  Care.  Three  questions 
were  used  to  determine  if  the  respondent  actually  experienced 
one  or  more  barriers  to  health  care  (table  1).  Respondents  were 
asked  if  they  had  a  doctor  (or  other  type  of  provider)  who  they 
saw  for  regular  health  care,  if  they  had  any  kind  of  health 
insurance  or  health  care  plan  and  if  there  was  a  time  during  the 
last  12  months  when  they  needed  to  see  a  doctor  but  could  not 
for  any  of  the  following  reasons:  cost,  lack  of  transportation, 
couldn't  get  appointment,  doctor  wouldn't  accept 
Medicaid/Medicare,  limited  hours  and/or  days  service  is 
available  or  another  reason  they  could  specify.  The  first  two 
questions  had  been  used  on  Missouri  BRFSS  and  earlier  special 
survey  instruments  and  the  third  question  was  developed  for 
this  survey. 

Table  1.  Determination  of  Barriers  to  Health  Care  Access 

♦  Do  you  have  a  doctor  who  you  see  for  regular  health  care? 

♦  Do  you  have  any  kind  of  health  insurance  or  health  care  plan? 

♦  Was  there  a  time  during  the  last  1 2  months  when  you  needed 
to  see  a  doctor  but  could  not  for  any  of  the  following  reasons? 

♦  Cost 

♦  Lack  of  transportation 

♦  Couldnt  get  an  appointment 

♦  Couldnt  find  a  provider  to  accept 

Medicaid/Medicare 

♦  Limited  hours  of  services  availability 

♦  Other  (specify) 


Analysis.         Simple     frequencies 


calculated     by 
Projections  for 

number  of  residents  impacted  were  based  on  1990  census  data 

for  Boone  County. 


were 
sociodemographic  groupings  using  SAS  13, 


Study  Area.  Boone  County  is  located  in  the  center  of  Missouri, 
midway  between  St.  Louis  and  Kansas  City.  The  county  has  a 
mixed  economy,  based  primarily  on  health  care,  institutions  of 
higher  education,  corporate  regional  offices,  light  industry  and 
farming.  The  1990  population  of  Boone  County  was  1 12,379.5 


Boone  County  is  experiencing  more  rapid  growth  than  the  state 
as  a  whole;  the  1994  population  estimate  was  121,500. 
Projections  to  the  year  2000  indicate  a  10  year  population 
growth  of  over  17%,  compared  with  a  4.2%  gain  for  Missouri 
overall.7  The  largest  city  is  Columbia,  with  a  1992  population 
of  73,000.  In  1992,  there  were  seven  other  incorporated 
municipalities  in  the  county,  with  populations  ranging  from 
under  200  to  over  3,500,  and  a  number  of  unincorporated 
hamlets.  Over  95%  of  Boone  County  households  had  at  least 
one  residential  telephone  in  1990  (map  1). 


Map  1.  Percent  Telephone  Coverage  Boone  Co.,  MO 


Based  on  1990 
Census  Data 


□   86%  to  90% 

1   90%  to  95% 

■    95%  to  100% 


According  to  MDOH,  Boone  County  has  36.3  full-time 
physician  FTEs  per  10,000  population,  while  the  state  of 
Missouri  has  16.2  full-time  physician  FTEs  per  10,000 
population.8  The  American  Medical  Association  reports  19.8 
patient-care  physicians  per  10,000  population  in  the  United 
States.9  Using  1990  census  data5  and  figures  published  by  the 
American  Hospital  Association,  Boone  County  has  142 
hospital  beds  per  10,000  population,  while  Missouri  and  the 
United  States  have  55.8  and  47.7  beds  per  10,000  population, 
respectively. 

Results 

The  897  survey  respondents  and  the  county  population,  based 
on  1990  census  data,  were  similar  with  respect  to  age,  income 
and  household  composition  (table  2).  African  Americans  and 
other  non-whites  constituted  7.5%  and  3.5%,  respectively,  of 
the  county  population  but  only  5.2%  and  3.0%,  respectively,  of 
the  survey  sample.  Survey  respondents  were  more  likely  to  be 
female  (58.9%  v.  51.6%),  college  educated  (41.7%  v.  36.5%) 
and  not  in  the  work  force  (36.6%  v.  26.3%)  than  the  county 
population  and  less  likely  to  be  employed  (58.2%  v.  65.9%), 
disabled  (1.9%  v.  4.6%)  or  have  less  than  a  high  school 
education  (8.8%  v.  15.2%).  Full-time  students  (129)  were 
removed  from  subsequent  analyses,  leaving  a  final  sample  of 
768  Boone  County  residents. 
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Table  2.  Characteristics  of  Sample  (n- 

=897)  and 

Boone  County  Population 

Percent 

Composition 

County 

Survey 

♦     Race 

White 

89.0 

91.5 

Black 

7.5 

5.2 

Other 

3.5 

3.0 

♦      Age 

18-24 

24.1 

21.9 

25-44 

45.4 

42.8 

45-64 

19.0 

20.0 

65  + 

11.5 

14.7 

♦      Income 

<  $10,000 

18.6 

21.5 

$10,000-$  19,999 

20.7 

19.0 

$20,000-$49,999 

41.5 

41.8 

>  $50,000 

19.2 

17.7 

♦      Education* 

Less  than  high  school 

15.2 

8.8 

High  school/GED 

25.3 

27.5 

Some  college 

23.0 

21.8 

College  or  higher 

36.5 

41.7 

♦      Sex 

Male 

48.4 

41.1 

Female 

51.6 

58.9 

♦     Employment 

Employed 

65.9 

58.2 

Unemployed 

3.2 

3.2 

Disabled 

4.6 

1.9 

Not  in  work  force 

26.3 

36.6 

♦     Household 

Married  couple  w/  kids 

24.3 

24.6 

composition 

Married  couple  no  kids 

25.5 

25.1 

Male  parent  w/  kids 

1.5 

1.7 

Female  parent  w/  kids 

6.5 

6.2 

Other 

42.2 

42.4 

Table  3.  Prevalence  of  Barriers  to  Health  Care  Among 
Respondents 


*Respondents  and  residents  age  25  or  more  for  educational  level. 

Measurements  of  barriers  to  care  can  be  categorized  as 
financial,  temporal  or  provider-related.  Financial  variables 
include  insurance  status  and  problems  with  the  cost  of  care. 
Variables  used  to  assess  temporal  barriers  to  care  include 
appointment  availability  and  limited  hours.  Provider-related 
issues  include  lack  of  access  to  a  regular  provider,  providers 
who  refuse  to  accept  Medicaid  or  Medicare  and  distance  or  lack 
of  transportation  to  a  provider. 

Fifty-two  percent  of  respondents  reported  experiencing  one  or 
more  barriers  to  health  care  during  the  preceding  12  months. 
Projecting  to  the  adult  population  of  Boone  County,  nearly 
43,000  adults  would  have  experienced  at  least  one  barrier  to 
health  care  during  the  period.  Table  3  shows  the  breakdown  by 
specific  variables.  The  first  column  indicates  the  percent  of 
respondents  reporting  a  particular  barrier.  The  middle  column 
represents  the  proportion  of  these  respondents  who  could  not 
obtain  medical  care  when  they  needed  it  for  any  reason  during 
the  previous  12  months.  Multiplying  column  one  by  column 
two  yields  the  percent  of  respondents  reporting  a  particular 
barrier  who  were  unable  to  obtain  care  during  the  preceding 
year.  For  example,  the  most  frequently  reported  barrier  was 
having  no  regular  provider,  reported  by  34%  of  respondents. 
However,  only  38%  of  respondents  with  no  regular  provider 
reported  problems  getting  care  when  it  was  needed,  leaving 
13%  of  respondents  without  a  regular  provider  who  were 
unable  to  get  care  when  they  needed  it. 


Percent 

Reporting 

Barrier 

Percent 
Lacking 

Care 

Percent 
Affected 

Financial  barriers 

♦    Cost 

16 

100 

16 

♦    No  health  insurance 

13 

62 

8 

Temporal  barriers 

♦    Appointment  availability 

11 

100 

11 

♦    Limited  service  hours 

11 

100 

11 

Provider-related  barriers 

♦    No  regular  provider 

34 

38 

13 

♦    Transportation 

4 

100 

4 

♦    No  Medicare/Medicaid 

3 

100 

3 

Cost  was  the  second  most  common  barrier,  reported  by  16%  of 
respondents,  including  31%  of  those  with  household  incomes 
below  the  federal  poverty  level,  24%  of  the  "near  poor"  (100- 
200%  of  poverty)  and  5%  of  those  with  incomes  at  least  twice 
the  federal  poverty  level.  All  respondents  who  reported  cost  as 
a  barrier  were  prevented  from  getting  medical  care  when  they 
needed  it  at  least  once  during  the  preceding  12  months.  Thus, 
the  percent  actually  affected  by  cost  remains  16%,  making  it  the 
most  prevalent  barrier  to  health  care  for  Boone  County  adults. 
Projected  to  the  adult  population  of  the  county,  13,000  Boone 
County  adults  would  have  experienced  cost  as  a  barrier  to 
getting  needed  care  in  the  preceding  year. 


Figure  1.  Boone  County  Adults 

Experiencing  Cost  as  a  Barrier,  by 

Age 


18-29 


30-49  50-64 

Respondent  Age  (yr) 


65+ 


Younger  adults  experienced  cost  as  a  barrier  to  a  greater  degree 
than  older  adults  (figure  1).  Of  individuals  who  were  out  of 
work,  nearly  62%  experienced  cost  as  a  barrier  (figure  2). 
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Figure  2.  Boone  County  Adults 

Experiencing  Cost  as  a  Barrier,  by 

Occupation 


Figure  4.  Boone  County  Adults 

Experiencing  Cost  as  a  Barrier,  by 

Education* 
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Notably,  approximately  17%  of  employed  respondents  also 
experienced  cost  as  a  barrier.  As  might  be  expected,  the  percent 
of  adults  experiencing  barriers  to  health  care  increased  as 
household  income  decreased  and  was  highest  for  respondents 


Figure  3.  Boone  County  Adults 

Experiencing  Cost  as  a  Barrier,  by 

Household  Income 


affected  by  the  cost  of  medical  care  include  African 
Americans  and  unemployed  persons.  Population  subgroups 
reporting  the  greatest  difficulty  with  cost  in  terms  of  both 
proportions  and  absolute  numbers  are  respondents  with 
annual  household  incomes  under  $20,000,  those  who  have 
children  under  18  years  of  age  and  those  who  have  not 
graduated  from  college. 

The  third  most  common  barrier  was  lack  of  health  insurance, 
reported  by  13%  of  respondents  (table  3).  Of  uninsured 
residents,  39%  had  been  without  coverage  for  at  least  one 
year  and  23%,  for  over  five  years. 
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with  annual  household  incomes  below  $20,000  (figure  3).  As 
educational  attainment  increased,  the  importance  of  cost  as  a 
barrier  decreased  (figure  4). 


The  population  subgroups  reporting  the  greatest  difficulty 
with  cost  in  absolute  numbers  are  white,  employed  and/or 
adults  aged  18  to  49  (table  4).    Groups  disproportionately 


Table  4.  Characteristics  of  Boone  County  Adults 
Experiencing  Cost  as  a  Barrier  to  Health  Care 


Survey  % 

County 

Affected 

Projection 

♦ 

Race 

♦      Caucasian 

15 

11,000 

♦      African-American 

25 

1,360 

♦ 

Employment 

♦      Employed 

17 

9,000 

♦      Unemployed 

62 

1,900 

♦ 

Age 

♦      18-49  years  old 

21 

12,500 

♦      >49 

7 

1,380 

♦ 

Annual  income 

♦       <$20,000/yr 

30 

5,000 

♦      $20,000  or  more 

10 

2,500 

♦ 

Household  composition 

♦      Family  with  children 

21 

2,800 

♦      Family  without  children 

13 

1,500 

♦ 

Education  (age  25  or  more) 

♦      Not  a  college  graduate 

19 

7,600 

♦      College  graduate 

11 

2,460 
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Appointment  availability  and  limited  hours  of  service  were 
each  reported  as  barriers  by  11%  of  respondents  (table  3).  Ten 
percent  of  respondents  who  used  the  ER  reported  they  did  so 
because  they  had  no  other  place  to  go  or  could  not  get  an 
appointment  with  their  usual  provider.  Respondents  with 
household  incomes  <  $20,000  were  over  twice  as  likely  to  have 
visited  the  ER  three  or  more  times  than  those  with  incomes  > 
$20,000  (5.4%  and  12.9%,  respectively). 

Discussion 

Access  to  health  care  is  a  major  problem  in  Boone  County, 
affecting  over  half  of  all  adults.  In  terms  of  actually  preventing 
individuals  and  families  from  getting  care  when  it  is  needed, 
cost  is  the  most  prevalent  barrier  to  health  care  access,  affecting 
16%  of  respondents  in  the  past  year.  Some  population 
subgroups  are  disproportionately  affected  by  the  cost  of  health 
care;  these  include  African  Americans,  unemployed  persons, 
young  families  with  children  and  respondents  with  lower 
household  incomes.  Parallel  to  the  above  finding,  a 
disproportionate  chronic  disease  morbidity  and  mortality 
burden  is  shared  by  these  same  population  subgroups.1,15,16,1 
These  two  findings  may  be  related.  Previous  research  has 
identified  that  part  of  the  disproportionate  burden  can  be 
attributed  to  lack  of  access  to  appropriate  and  timely  health 

.  1,15,16,18,19 

care. 


solely  to  estimate  barriers  to  access  among  minorities  but  to 
give  a  good  representation  to  the  county  as  a  whole. 

Since  relatively  few  population-based  local-level  studies 
focusing  on  access  to  health  care  have  been  reported,  this  study 
can  provide  some  useful  insights  to  other  researchers.  As 
managed  care  comes  to  mid-Missouri,  this  study  can  provide 
baseline  data  and  serve  as  a  model  for  continued  measurement 
of  access  to  care,  utilization  and  need. 
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This  study  has  a  number  of  strengths.  It  is  a  population-based 
study  focusing  on  a  single  county.  Existing  standardized 
questions  were  supplemented  with  locally-developed  or 
adapted  questions.  Interviewers  were  trained  and  experienced 
in  collecting  behavioral  risk  factor  data.  An  excellent  response 
rate  was  achieved.  Study  results  provide  a  good  basis  for  health 
care  planning  at  the  local  level. 

The  study  also  has  a  few  weaknesses.  County  residents  without 
a  telephone  were  ineligible  for  inclusion;  however,  less  than  5% 
of  households  in  the  county  had  no  telephone.  Access  and 
barriers  were  self-reported  and  were  based  in  part  on  responses 
to  newly  developed  questions  that  had  been  tested  and  piloted 
but  had  not  undergone  extensive  testing  for  validity  or 
reliability.  In  two  published  studies  that  have  attempted  to 
measure  validity,20,21  substantial  agreement  was  shown  for  most 
cardiovascular  risk  factors  between  BRFSS-system  collected 
data  and  physiologic/in-person  interview  data.  Two  other 
studies,  one  conducted  in  New  York  City  and  one  by  MDOH 
BRFSS  staff,  have  measured  reliability.  The  findings  of  the 
two  studies  generally  agree;  in  items  comparable  between  the 
New  York  study  and  the  Missouri  study,  the  Missouri  estimates 
of  reliability  were  slightly  higher.23  The  Missouri  study  found 
no  clear  pattern  in  reliability  based  on  whether  BRFSS 
questions  were  developed  at  CDC  or  in  Missouri.  Missouri 
results  indicate  that  reliability  may  be  higher  for  behavioral 
variables  and  lower  for  knowledge  and  attitudinal  variables.23 
All  three  key  questions  used  to  measure  access  to  care  related  to 
behaviors  rather  than  knowledge  or  attitudes.  African 
Americans,  particularly  African  American  males,  were  under- 
represented  in  the  sample,  as  were  individuals  with  less  than  a 
high  school  education.  Nevertheless,  we  were  able  to  identify 
differences  by  socioeconomic  and  ethnic  groups  despite  this 
weakness.    In  addition,  the  purpose  of  our  research  was  not 
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VARIATIONS  IN  WELL-CHILD  VISIT  RATES  BY  MEDICAID  ELIGIBILITY  STATUS 

Elicia  J.  Herz,  The  MEDSTAT  Group, 
Norma  I .  Gavin  and  Kate  Sredl 
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Study  Objective  and  Background 

Under  a  three-year  HCFA-funded  project,  the 
MEDSTAT  Group  is  investigating  the  impact  of 
OBRA-89  legislative  changes  on  the  provision  of 
preventive  and  curative  services  to  Medicaid 
children  both  within  and  outside  State  EPSDT 
programs.  By  fiscal  year  1995,  all  States  are 
expected  to  increase  participation  in  EPSDT  to  80 
percent  of  all  children  enrolled  in  Medicaid. 

Overall  baseline  analyses  of  1989  data  indi- 
cate that  only  41  to  51  percent  of  Medicaid  chil- 
dren recommended  to  have  well-child  visits  did 
so,  and  only  30  to  44  percent  of  recommended 
visits  were  made.  Medicaid  children  as  a  whole 
were  receiving  well-child  care  at  rates  only 
slightly  lower  than  children  with  private  health 
insurance . 

The  objective  of  the  analyses  described  in 
this  report  was  to  investigate  variations  in  the 
receipt  of  well-child  visits  among  Medicaid  chil- 
dren by  eligibility  category.  Medicaid  children 
in  different  eligibility  categories  vary  in  many 
respects,  including  for  example,  age  distribu- 
tion, Medicaid  enrollment  duration,  health  sta- 
tus, financial  resources  and  EPSDT  outreach 
activities.  These  characteristics  may  influence 
parents'  decisions  to  seek  health  care  for  their 
offspring  and  thus  receipt  of  services  such  as 
well-child  visits.  Results  from  these  analyses 
will  help  target  new  policy  and  programmatic 
initiatives  to  the  groups  of  children  with  the 
greatest  need  for  preventive  services. 

Methodology 

The  EPSDT  participant  and  screening  ratios 
reported  to  HCFA  by  the  States  in  1989  were  not 
computed  in  a  consistent  manner,  nor  were  they 
believed  to  provide  accurate  measures  of  States' 
success  in  reaching  and  screening  Medicaid  chil- 
dren. In  particular,  the  ratios  neglected  to  take 
the  periodicity  schedule  for  screening  visits 
relevant  to  the  child' s  age  and  length  of 
Medicaid  enrollment  into  consideration  and 
counted  children  in  continuing  care  arrangements 
as  having  received  all  screens  whether  or  not 
they  actually  did  so. 

To  overcome  these  measurement  problems,  we 
developed  new  methods  to  more  accurately  assess 
the  receipt  of  Medicaid-f inanced  well-child  ser- 
vices rendered  both  within  and  outside  formal 
EPSDT  programs.  To  do  so,  we  assigned  two  weights 
to  each  child  in  the  database-a  participation 
weight  and  a  compliance  weight.  The  participation 
weight  (P)  reflects  the  child's  expected  prob- 
ability of  being  a  user  of  preventive  care  visits 
during  the  year  while  the  compliance  weight  (S) 
reflects  the  child's  expected  number  of  preven- 
tive care  visits  during  the  analysis  year.  We 
adjusted  the  values  assigned  for  both  weights  for 
the  child's  age  and  enrollment  duration  and  the 
American  Academy  of  Pediatrics  (AAP)  screening 
periodicity  schedule. 

In  deriving  the  weights,  we  first  determined 
the  recommended  number  of  screening  visits  for  a 
child  enrolled  for  the  full  12  months  of  1989 
based  on  the  1989  AAP  periodicity  schedule  and 


the  age  of  the  child  at  the  end  of  the  year.  For 
younger  children,  the  recommended  number  of 
screening  visits  was  determined  for  each  month  of 
age.  For  children  in  age  groups  for  which  only 
one  screening  visit  was  recommended  every  other 
year,  the  child  was  assumed  to  be  equally  likely 
to  have  the  screening  visit  anytime  during  the 
two  years.  Therefore,  the  recommended  number  of 
screening  visits  was  one-half  (0.5)  visit;  that 
is,  the  probability  of  an  event  occurring  in 
either  of  two  periods  under  a  uniform  distribu- 
tion function  over  the  two  periods. 

We  then  adjusted  for  duration  of  enrollment 
by  multiplying  the  number  of  recommended 
screening  visits  by  the  fraction  of  the  year  that 
the  child  was  enrolled,  or  if  the  child  was  less 
than  12  months  of  age,  the  fraction  of  the 
child's  life  during  which  s/he  was  enrolled.  This 
methodology  assumes  that  a  child  was  equally 
likely  to  receive  a  screening  visit  during  a 
month  in  which  s/he  was  enrolled  as  during  a 
month  in  which  s/he  was  not  enrolled.  Thus,  the 
expected  number  of  screening  visits,  S..,  for  the 
ith  child  in  the  jth  age  group  is: 


_     Months  Enrolled, 

S  = 

,J     Months  of  Life,. 


x  No.  of  Recommended  Visits; 


where  months  of  life,,  is  12  for  all  children 
except  infants.  The  child's  compliance  weight  is 
simply  the  expected  number  of  screening  visits, 
S  .  The  participation  weight,  the  probability  of 
the  child  being  a  user  of  preventive  care  visits, 
is  equal  to  one  if  the  expected  number  of 
screening  visits  for  the  child  is  greater  than  or 
equal  to  one.  Otherwise, _the  child's  participa- 
tion weight  is  equal  to  S  ,  that  is: 


If  S„  >  1  then  PM  =  1; 


else  P„  =  SB. 


We  used  these  weights  to  compute  participa- 
tion and  compliance  rates  for  children  in  dif- 
ferent age  groups.  Participation  rates  give  the 
percentages  of  children  with  at  least  one  screen- 
ing visit  among  those  recommended  (expected)  to 
have  at  least  one  screening  visit.  The  numerator 
for  the  participation  rate  is  the  count  of  indi- 
viduals with  any  preventive  care  visits  during 
the  year  (i.e.,  P   =  1  for  children  with  at 
least  one  preventive  care  visit  and  zero  for 
children  with  no  such  visits) .  The  denominator  is 
the  total  expected  number  of  participants, 
computed  by  summing  the  participation  weights 
over  the  child  population  being  tabulated. 


Participation  Rate  = 


Actual  No.  of  Participants 
Expected  No.  of  Participants 


IP, 
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Compliance  rates  give  the  percentages  of 
total  recommended  (expected)  screening  visits 
children  in  different  subgroups  actually  had.  The 
numerator  of  the  compliance  rate  is  the  total 
number  of  screening  visits  children  had  during 
the  year  (i.e.,  Sn) .  The  denominator  is  the 
total  expected  number  of  screening  visits,  com- 
puted by  summing  children's  compliance  weights. 


Compliance  Rate  = 


Actual  No.  of  Screens 


Ss. 


ExpectedNo.  of  Screens 


Ss. 


Data  Source  and  Study  Population 

The  data  used  in  this  analysis  were  drawn 
from  the  1989  Medicaid  Tape-to-Tape  Database 
which  includes  complete  enrollment  and  claims 
information  for  four  States— Calif ornia,  Georgia, 
Michigan  and  Tennessee.  For  each  of  these  States, 
all  Medicaid  enrollees  under  the  age  of  21  years 
were  included  in  the  analysis,  excluding  those: 
(1)  enrolled  in  capitated  programs,  (2)  residing 
in  institutions,  and  (3)  with  dual  Medicare  and 
Medicaid  coverage.  For  these  analyses,  children 
were  classified  into  one  of  five  groups  based  on 
the  eligibility  category  under  which  they  were 
enrolled  for  the  greatest  number  of  months  during 
1989:  (1)  AFDC  cash  assistance,  (2)  SSI  blind/ 
disabled,  (3)  foster  care,  (4)  poverty-related 
expansion,  and  (5)  medically  needy/other. 

Results  and  Conclusions 

Among  the  five  eligibility  categories  inves- 
tigated, foster  care  children  had  the  highest 
rates  of  well-child  visit  use,  while  SSI  blind/ 
disabled  generally  had  the  lowest  such  rates. 
Across  States,  the  percentage  of  foster  care 
children  recommended  to  have  a  visit  and  who 
actually  had  at  least  one  such  visit  ranged  from 


54  to  64  percent.  Similar  figures  for  the  SSI 
blind/disabled  ranged  from  26  to  34  percent  (see 
Figure  1).  In  addition,  foster  care  children  had 
44  to  62  percent  of  AAP  recommended  visits  Ln 
1989,  compared  to  29  to  37  percent  among  SSI 
blind/disabled  children  (see  Figure  2). 

The  high  rates  of  well-child  visit  use  among 
foster  care  children  may  be  explained  in  part  by 
foster  parents'  contractual  obligations  to  attend 
to  their  foster  children's  health  care  needs.  The 
low  rates  among  SSI  blind/disabled  children  may  be 
explained  by  the  need  for  on-going  physician  care 
to  treat/manage  their  disabling  conditions  which 
may  preclude  the  receipt  of  separately  billed,  and 
thus  identifiable,  preventive  visits. 

To  further  investigate  the  possibility  that 
SSI  blind/disabled  children  may  be  underserved 
with  respect  to  preventive  services,  we  also  com- 
pared eligibility  groups  on  immunization  compli- 
ance rates  (using  similar  computation  methods  as 
described  above  for  preventive  care  visits) .  This 
analysis  was  based  on  the  assumption  that  physi- 
cians separately  bill  for  administration  of  immu- 
nizations and  therefore,  that  these  immunizations 
would  be  found  in  the  claims  database. 

The  data  for  all  children  show  that  immuniza- 
tion rates  for  SSI  blind/disabled  children  were 
lower  than  similar  rates  for  other  Medicaid  chil- 
dren (see  Figure  3),  suggesting  that  in  1989,  SSI 
blind/disabled  children  as  a  whole  may  have  been 
underserved  with  respect  to  preventive  services 
vis-a-vis  other  Medicaid  children.  For  the  subset 
of  children  with  EPSDT  visits,  however,  there  ap- 
pear to  be  few  differences  in  immunization  rates 
across  eligibility  groups  (see  Figure  4) .  Overall, 
these  analyses  point  out  the  need  for  further  in- 
vestigation into  the  reasons  why  SSI  blind/disabled 
children  receive  fewer  preventive  care  services 
than  other  groups  of  Medicaid  children  and  how  best 
to  provide  such  care  to  this  subgroup. 


Figure  1 :  Preventive  Care  Visits : 
Participation  Rates 


Figure  2:  Preventive  Care  Visits: 
Compliance  Rates 
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Figure  3 :  Age-Appropriate 
Immunization  Compliance  Rate 


Figure  4 :  Percentage  of  Children 

with  EPSDT  Visits  Who  Were  Up-to-Date 

in  their  Childhood  Immunizations 


AFDC 


Foster 


Poverty 


ill     M 

AFDC  SSI  Foster  Poverty  Other 


DCA 
■  TN 


301 


IMPROVING  PREVENTIVE  SERVICES  DELIVERY  IN  A  RURAL  HEALTH  CARE  CLINIC 

Charles  H.  Darby,  Mayo  Clinic,  Mayo  Foundation 
James  M.  Naessens,  Susan  E.  Majerus,  Gregory  L.  Angstman,  Mary  E  Campion 
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Many  methods  have  been  tried  to  increase 
the  use  of  preventive  health  services  .  Mass 
media  appeals,  health  fairs,  mobile  outreach 
units  and  other  methods  have  been  shown  to 
have  low  cost-effectiveness  in  predominantly 
rural  areas.  This  paper  will  discuss  a  simple 
technique  used  at  Mayo  Kasson  Family  Practice 
Clinic  in  Kasson  Minnesota,  and  the  initial 
research  results  on  improving  the  delivery  of 
preventive  services  there. 

Kasson  Minnesota  has  approximately  3,500 
residents,  the  surrounding  county  about  double 
that  number.  Kasson  is  located  15  miles  west 
of  Rochester,  Minnesota  which  is  home  to  the 
Mayo  Clinic  and  a  large  IBM  manufacturing 
facility.  Both  communities  are  approximately 
70  miles  south  of  the  Minneapolis-St .  Paul 
metroplex. 

Kasson  is  a  predominantly  an 
agricultural  community.  Some  light  industry 
and  other  regional  employers  complete  the 
economic  base  for  the  area. 

The  Kasson  Family  Practice  Clinic 
(KMFPC)  includes  ten  staff  family 
practitioners,  an  appropriate  complement  of 
registered  and  licensed  professional  nurses, 
lab  personnel,  as  well  as  reception,  records, 
and  secretarial  employees.  In  addition  to 
these  permanent  members  of  the  staff,  KMFPC 
hosts  25  residents  in  family  practice  from  the 
Mayo  Health  System.  These  residents  rotate 
through  the  facility  annually,  usually 
arriving  in  late  summer. 

The  specific  preventive  services  we 
studied  included:  1) smoking  cessation 
assistance;  2)  adult  immunizations: 
pneumococcal  pneumonia,  influenza,  and 
tetanus-diphtheria;  3)  hypertension  screening; 
4)  cholesterol  screening;  5)  pap  smear 
screenings  for  cervical  cancer;  6)  self- 
administered  and  clinical  breast  examinations; 
and  7)  mammography  screening.   This  paper  will 
describe  our  methods,  descriptive  findings, 
prevalence  findings  and  conclusions . 

Methods : 

All  patients  18  and  older  who  received 
medical  care  at  the  Kasson  Clinic  in  1993 
(N=20,371)  and  in  the  last  three  months  of 
1994  (N=4,171)  were  identified  from  Mayo ' s 
electronic  billing  data.   A  random  sample  of 
patients  was  selected  for  each  period 
identified.  The  medical  records  of  these 
patients  were  abstracted  by  registered  nurses 
to  determine  the  level  of  documentation  of 
each  preventive  service  and  the  prevalence  of 
each  disease  precursor  identified  above. 

Upon  completion  of  our  abstracting  of 
the  1993  baseline  sample  (n=249) ,  a  Medical 
Record  Form  (MRF)  of  preventive  health  care 
questions  was  introduced  at  the  Clinic.   The 
form  is  illustrated  here  as  Figure  1.  The 
MRF's  questions  were  addressed  by  the  rooming 


nurse  to  each  arriving  patient.  The  primary 
care  physician  who  examined  the  patient  used 
the  MRF  as  a  status  summary  of  the  preventive 
services  needed  by  and  provided  for  the 
patient . 

We  extracted  a  subsequent  simple  random 
sample  (n=140)  from  the  fourth  quarter  1994 
encounters  at  the  same  facility.  This  sampling 
allowed  us  to  study  the  effects  of  the  MRF 
during  the  first  three  months  of  its  use  in 
the  clinic.   Both  samples  were  subjected  to 
the  same  analyses .  Table  1  presents  a 
comparison  of  the  two  samples  pre-  and  post- 
MRF. 

Comparisons  were  made  on  the  percent  of 
patients  with  documented  services  between  the 
two  periods  using  chi-square  analyses  to 
assess  the  effectiveness  of  the  MRF.   A 
multivariate  analysis  adjusting  for  age  and 
sex  based  on  logistic  regression  was  also 
performed.  Additional  analysis  was  conducted 
to  assess  whether  patients  of  a  particular  age 
group  (18-39,  40-64,  65+)  were  more  likely  to 
have  a  screened  disease  (e.g.  hypertension, 
excessive  cholesterol)  or  more  likely  to 
receive  selected  services.   Trends  in  the 
adult  immunization  rates  shown  in  Figure  4 
were  based  on  the  total  number  of  each 
immunization  provided  at  the  Kasson  Clinic  for 
1988  through  1994. 

Basic  Improvement  Results 

Introduction  of  the  new  MRF  for 
preventive  services  demonstrated  success. 
Documentation  improved  for  adult  immunization 
rates  and  cholesterol  counseling  by  medical 
staff.  There  was  improved  diagnosis  of 
hypertension  in  the  presence  of  elevated  blood 
pressures.  In  fact,  documentation  rates 
improved  for  ten  of  the  eleven  items  studied. 
The  rates  of  improvement  which  were  noted 
between  the  baseline  and  subsequent  samplings 
are  shown  in  Figure  2 . 

Improvements  were  not  limited  to 
documentation.   The  delivery  of  preventive 
services  also  increased. 

Adult  immunization  rates  increased 
dramatically  (Figure  3).  Diagnosis  of 
hypertension  was  improved  by  5 . 3  percent 
(89.6%  to  94.9%).   Counseling  for  cholesterol 
increased  by  14.8  percent  (29.3%  to  44.1%). 
The  rate  of  women  having  a  clinical  breast 
examination  increased  by  3.2  percent  across 
the  two  samples . 

Diphtheria-tetanus  immunizations 
increased  from  35  per  month  in  1992  to  over  70 
per  month  in  1994.   Influenza  vaccinations 
increased  from  235  per  year  in  1988  to  1,262 
per  year  in  1993  and  1,161  in  1994.   Medical 
record  review  in  1993  showed  that  93  percent 
of  elderly  patients  at  the  clinic  during  that 
year  received  the  influenza  vaccine.  The 
number  of   pneumococcal  vaccines  increased 
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from  33  per  year  in  1988  to  310  in  1994.  Over 
60  percent  of  the  elderly  clinic  patients 
received  pneumococcal  vaccinations.  It  is 
noteworthy  that  two  of  three  immunization 
counts  demonstrated  their  largest  single  year 
increase  in  the  year  the  new  preventive 
services  Medical  Record  Form  was  introduced. 
See  Figure  4  for  the  trend  in  these 
vaccinations . 

Prevalence  Findings 

Combining  the  two  samples  we  examined 
the  prevalence  of  medical  conditions  and 
services  provided  patients  in  both  time 
periods.  Three  such  findings  found  to  be 
statistically  significant  are  highlighted. 

First,  women  65  and  older  were  three 
times  less  likely  to  have  had  a  pap  smear  in 
the  most  recent  three  years  than  women  in  the 
40  to  64  age  group  (p-value  =0.007). 

Second,  older  adults  (ages  65  and  over) 
were  more  likely  to  describe  themselves  as 
lifetime  non-smokers  than  were  adults  in  the 
18-39  or  40-64  age  group  (p-value  =0.012). 

Lastly,  senior  men  and  women  (in  the  65+ 
age  group)  were  more  likely  to  have  abnormally 
elevated  total  cholesterol  readings  than 
either  other  age  class  (p-value<0 . 001 ) 


Conclusions 

We  have  confirmed  three  principles  by 
beginning  this  review  process.   First,  it  is 
apparent  that  simple  medical  record  tools  can 
assist  physicians  and  nurses  in  addressing  the 
preventive  care  needs  of  their  patients.  By 
periodically  reviewing  and  analyzing  such 
data,  we  will  increase  our  understanding  of 
rural  practice  and  hence  provide  a  better 
focus  for  our  improvement  efforts. 

Secondly,  documentation  of  preventive 
services  consideration  and  delivery  can  be 
accomplished  in  a  time-efficient  manner  by 
using  such  a  Medical  Record  Form.  The 
preprinted  questions  insure  uniformity  of 
application. 

Lastly,  reducing  variation  in  preventive 
service  screening  will  promote  the  best  use  of 
our  vital  and  limited  rural  health  resources. 
We  will  continue  to  serve  a  growing  rural 
community  by  improving  our  care  delivery 
processes . 

As  a  result  of  this  successful 
implementation,  additional  primary  care  sites 
in  the  Mayo  Health  System  have  implemented 
this  MRF.  Eventually,  it  is  expected  that  the 
MRF  will  become  available  to  all  MHS 
practitioners  via  the  Mayo  Electronic  Medical 


Figure  1:   Implemented  Medical  Record  Form  (MRF) 


Name: 


Clinic#: 
D.O.B. 


PREVENTION 

1994  1995  1996 


1994 

1995 

1996 

Yes 

No 

Yes 

No 

Yes 

No 

Blood  Pressure 

Have  you  had  your  blood  pressure  checked  in  the  last 
year? 

Cholesterol 

Have  you  had  your  cholesterol  checked  in  the  last  5 
years?             If  yes,  which  year? 

Were  previous  levels  acceptable? 

Smoking 

Do  you  smoke? 

Do  you  wish  to  quit  smoking? 

Immunizations 

Have  you  had  a  tetanus  immunization  in  the  last  10 
years?            If  yes  which  year? 

Children 

Immunization  status  reviewed 

Up-to-date  for  current  year? 
Check  yes  when  complete  for  year. 

Persons  65  and  older 

Have  you  had  a  pneumococcal  (pneumonia)  vaccination 
in  your  lifetime? 

Do  you  need  an  influenza  vaccination  this  year? 

WOMEN  ONLY 

Do  you  know  how  to  perform  a  Breast  Self  Exam? 

Do  you  perform  monthly  Breast  Self  Exams? 

Have  you  had  a  Breast  Exam  in  the  last  year? 

Have  you  had  a  cervical  pap  smear  in  the  last  3  years? 

WOMEN  AGE  40  -  50  ONLY 

Have  you  had  a  mammogram  in  the  last  2  years? 

WOMEN  AGE  50  AND  OVER 

^Have  you  had  a  mammogram  in  the  last  year? 
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Record  (EMR) .  Phased  introduction  of  the  EMR 
is  scheduled  to  begin  late  in  1995. 

Identifying  and  understanding  barriers 
to  providing  preventive  services  in  a  primary 
care  setting  will  help  us  to  further  improve 
the  care  we  provide  our  patients,  and 
ultimately  potentially  decrease  the  incidence 


and  severity  of  disease  in  the  regional 
population.  Payors  and  patients  alike  expect 
continuous  improvement  in  the  quality  of  care 
we  deliver  in  rural  settings.  Consistently 
measuring  our  efforts  will  reaffirm  our 
commitment  to  their  expectations. 


Table  1:   Comparison  of  Disease  Prevention/Detection/Management  Results 
Baseline  and  Post  MRF  Analyses 


Smoking  Status 

Not  documented 

Non-smoker 

Past  smoker 

Current  smoker 
Smoking  Advice 

Not  documented 

Patient  refused 

MD/RN  documented  quitting  advice 
Smoking  Management 

Not  documented 

Nonspecific  documentation 

Patient  self  management 

Written  materials  provided  to  patient 

Nicotine  gum  ordered 

Nicotine  patch  ordered 

Smoking  cessation  program  referral 

Any  2  or  more  of  prior  4  choices 

Other  management  effort 
Smoking  Results 

No  documentation 

Smoking  stopped 

Smoking  reduced 

Smoking  unchanged 
Cholesterol 

Documentation  of  screening  in  most  recent  5  years 

Screening  result  was  normal 

Screening  result  was  abnormal  (total  cholesterol  >  242) 

Patient  declined  or  done  more  than  5  years  ago 
Cholesterol  Test  Result  Follow-up 

No  documentation  or  no  follow-up 

Patient  counseled  by  MD/RN 

Patient  referral  to  dietitian 

Patient  given  written  material 

Any  of  prior  3  choices 
BP  Screening 

Not  documented 

Not  hypertensive  (BP<  139/89) 

Patient  diagnosed  hypertensive 

BP  >  1 40/90  but  not  documented  hypertensive 
Hypertensive  Management 

No  documentation 

Counseling  only 

Counseling  and  medications 

Patient  declined  treatment 
Immunizations: 
Pneumococcal 

Not  documented 

Not  applicable 

Given 

Patient  declined 


Baseline 

Subsequent 

Baseline 

Subsequent 

n(%) 

n(%) 

Influenza 

n(%) 

n(%) 

13  (5.2) 

3  (3.1) 

Not  documented 

184  (73.9) 

21  (21.4) 

135  (54.2) 

78  (79.6) 

Not  applicable 

0  (0.0) 

35  (35.7) 

49  (19.7) 

5(5.1) 

Given 

58  (23.3) 

32  (32.7) 

52  (20.9) 

12(12.2) 

Patient  declined 
DT 

Not  documented 

7  (2.8) 

10(10.2) 

22  (42.3) 

1  (8.3) 

127  (51.0) 

20  (20.4) 

1  (19) 

6  (50.0) 

Not  applicable 

0  (0.0) 

2  (2.0) 

29  (55.8) 

5(41.7) 

Given 

100  (40.2) 

67  (68.4) 

Patient  declined 

21  (8.4) 

6(6.1) 

31  (59.6) 

2(16.7) 

Immunization  Education 

1  (19) 

0  (0.0) 

Not  documented 

235  (94.4) 

52(53.1) 

6(11.5) 

10(83.3) 

Not  applicable 

12  (4.8) 

2  (2.0) 

2  (3.9) 

0  (0.0) 

Given 

2  (0.80) 

20  (20.4) 

2  (3.9) 

0  (0.0) 

Patient  declined 

0  (0.0) 

24  (24.5) 

1  (19) 

0  (0.0) 

Pap  Smear  Screening 

5  (9.6) 

0  (0.0) 

No  documentation 

32  (23.0) 

8(11.8) 

Done  <  3  years  ago  at  Mayo 

93  (67.0) 

51  (75.0) 

4  (7.7) 

0  (0.0) 

Done  >  3  years  ago  at  Mayo 

11  (7.9) 

8(11.8) 

Patient  report  done  <  3  years  ago  elsewhere 

1  (0.7) 

1  (1.5) 

24  (46.2) 

6  (50.0) 

Patient  report  done  >  3  years  ago  elsewhere 

2(1.4) 

0  (0.0) 

3  (5.8) 

0  (0.0) 

Pap  Smear  Results 

4  (7.7) 

0  (0.0) 

No  documentation 

39  (28.1) 

10(14.7) 

21  (40.4) 

6  (50.0) 

Mayo  normal 

96  (69.1) 

55  (80.9) 

Mayo  abnormal  with  follow-up 

3  (2.2) 

0  (0.0) 

143  (57.4) 

83  (84.7) 

Mayo  abnormal  without  follow-up 

90(36.1) 

49  (50.0) 

Patient  report  normal  elsewhere 

0  (0.0) 

3  (4.4) 

50(20.1) 

19  (19.4) 

Patient  report  abnormal  elsewhere 

3(1.2) 

15  (15.3) 

Other 
Clinical  Breast  Exam 

1  (0.7) 

0  (0.0) 

75  (54.6) 

52(41.0) 

No  documentation 

44(31.7) 

7  (10.3) 

41  (29.7) 

30  (30.6) 

Patient  declined 

3  (2.2) 

12  (17.7) 

2  (1.4) 

0  (0.0) 

Done  in  most  recent  1 2  months 

63  (45.3) 

33  (48.5) 

2  (1.4) 

5(5.1) 

Done  between  1 3  and  36  months  prior  to  visit 

21  (15.1). 

10(14.7) 

18(13.0) 

11  (11.2) 

Done  more  than  36  months  prior  to  visit 

7  (5.0) 

5  (7.4) 

Not  applicable 

1  (0.7) 

1  (15) 

21  (8.4) 

1  (10) 

Breast  Self  Exams 

134(53.8) 

68  (69.4) 

No  documentation 

121  (87.1) 

10(14.7) 

68  (27.3) 

24  (24.5) 

Patient  instructed  in  office  visit  by  MD/RN 

14(10.1) 

51  (75.0) 

26  (10.4) 

5  (5.1) 

Patient  instructed  in  BSE  class/program 

3  (2.2) 

0  (0.0) 

Patient  declined  or  did  not  attend  class 

1  (0.7) 

7(10.3) 

21  (22.3) 

0  (0.0) 

Abnormal  Mammography  Findings 

12(12.8) 

5(17.2) 

No  documentation 

71  (51.1) 

22  (32.4) 

Normal  -  follow-up  documented 

4  (2.9) 

6  (8.8) 

61  (64.9) 

24  (82.8) 

Abnormal  -  follow-up  documented 

45  (32.4) 

28  (41.2) 

Normal  -  no  follow-up 

17  (12.2) 

12  (17.7) 

Abnormal  -  no  follow-up 

2  (1.4) 

0  (0.0) 

37  (14.9) 

30  (30.6) 

Breast  Self  Exam 

173  (69.5) 

28  (28.6) 

No  documentation 

121  (87.1) 

10(14.7) 

37  (14.9) 

33  (33.7) 

MD/RN  instructed  patient 

14(10.1) 

51  (75.0) 

2  (0.80) 

7  (7.1) 

Patient  attended  class  for  instruction 

3  (2.2) 

0  (0.0) 

Patient  declined  /  patient  did  not  attend  class 

1  (0.7) 

7(10.3) 
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Figure  2:    Documentation  Improvement  in  the  Patient  Medical  Record 
from  Baseline  to  Post  MRF  Implementation. 


Smoking  advice 
Smoking  management 
Cholesterol  screening 
BP  screening 
Influenza 
DT 
Immunization  education 
Pap  results 
Clinical  breast  exam 
Mammogram  findings 


Figure   3:     Adult    (Age   18   and  Over)    Immunization  Rate   Improvement 
from  Baseline    to    Post   MRF    Implementation. 


Pneumococcal 
pneumonia 

Influenza 


Figure  4:   Trends  in  Vaccinations  Administered 

Mayo-Kasson  Family  Practice  Clinic,  1988-1994 


1,400  -- 
1,200  -• 
1,000  -- 
800  -- 
600  ■- 
400  -- 
200 


■Diptheria  Tetanus 
•Pneumococcal  Pneumonia 
■Influenza 


1988 


1989 


1990 


1991 


1992 


1993 


1994 


305 


B 


'M 


at* 


Session  R 


persons 

with  AIDS 


iT 


k"-wv* 


-«~.|,...^ — i ^ i^ t     | 

rKPSf 

3  $  : 


L. 


RS 


M 


I 


■ 


THE  NATIONAL  DATABASE  FOR  HIV-INFECTED  AMERICAN  INDIANS,  ALASKA  NATIVES, 

AND  NATIVE  HAWAIIANS 

David  D.  Barney,    National  Native  American  AIDS  Prevention  Center 


In  September  1994,  the  National  Native 
American  AIDS  Prevention  Center  (NNAAPC) 
developed  a  national  database  composed  of 
information  about  American  Indians,  Alaska 
Natives,  and  Native  Hawaiians  who  are  infected 
with  HIV.  At  NNAAPC,  the  database  is  referred  to 
as  the  "national  database"  since  data  are  gathered 
about  individuals  from  many  diverse  geographic 
regions  and  tribal  communities  throughout  the 
United  States. 

History  of  NNAAPC 

NNAAPC  was  founded  in  1988  by  a  grant 
from  the  Centers  for  Disease  Control  and 
Prevention  (CDC).  The  mission  of  the  agency  has 
been  to  stop  the  spread  of  HIV  and  related  diseases, 
including  sexually  transmitted  diseases  and 
tuberculosis,  among  American  Indians,  Alaska 
Natives,  and  Native  Hawaiians  by  improving  their 
health  status  through  empowerment  and  self- 
determination.  Programming  efforts  at  that  time 
were  exclusively  prevention  oriented.  However,  in 
1991,  NNAAPC  was  awarded  a  grant  from  the 
Health  Resources  and  Services  Administration 
(HRSA)  to  operate  a  case  management 
demonstration  project  for  HIV-infected  American 
Indian  people  in  the  state  of  Oklahoma. 
Subsequently,  HRSA  has  expanded  the  capability  of 
NNAAPC  to  serve  tribal  communities  by  facilitating 
the  development  of  twelve  local  community-based 
programs  nationwide.  These  programs  serve  as 
data  collection  sites  for  NNAAPC  research  and 
model  development  activities. 

Composition  of  the  Database 

Prior  to  the  development  of  this  database, 
there  was  no  data  systematically  collected 
exclusively  for  the  purpose  of  describing  or 
monitoring  of  health  of  HrV-infected  American 
Indian,  Alaska  Native,  or  Native  Hawaiian 
individuals.  Thus,  the  national  database  is  the  first 
effort  to  identify  the  health  characteristics  and 
service  needs  specifically  for  these  populations. 
Until  this  database  was  created,  there  was  no 
consolidated  source  of  information  to  describe  the 
demographic  characteristics  of  those  American 
Indians,  Alaska  Natives,  or  Native  Hawaiians 
infected  with  HIV.  At  present,  the  national 
database  serves  as  a  valuable  research  tool  to 
address  basic  HrV-care  related  questions  about 
these  populations.  The  database  will  also  be 
essential  for  future  inquiry  into  more  complex 
research  questions.  However,  the  main  purpose  of 
the  database  has  been  to  develop  the  baseline 
information  that  is  essential  for  health  planners  and 


policy  makers  to  address  the  health  and  medical 
needs  of  these  at-risk  populations. 

The  database  is  comprised  of  two  types  of 
variables.  First,  there  are  35  variables  that  describe 
the  demographics  of  those  individuals  in  the 
database.  Some  of  these  demographic  variables  also 
serve  as  baseline  data  for  longitudinal  analyses. 
Second,  there  are  319  variables  that  monitor  an 
individuals'  health  status  over  a  two  year  period. 
These  variables  are  collected  on  a  bi-monthly  basis. 

As  of  July  1995,  the  national  database  has  a 
total  of  238  cases.  These  cases  increase  monthly  as 
new  cases  are  continuously  added  to  the  database. 
The  sample  itself  is  a  non-representative 
convenience  sample  composed  of  clients  from  eleven 
NNAAPC-sponsored  HPV-care  case  management 
programs.  All  of  these  programs  are  funded  by 
NNAAPC.  The  largest  program,  the  Ahalaya 
Project,  is  located  in  the  state  of  Oklahoma  with 
offices  in  Tulsa  and  Oklahoma  City.  The  other  sites 
include  programs  in  the  states  of  Alaska,  Hawai'i, 
North  Carolina,  New  York,  Wisconsin,  Washington, 
Arizona,  Kansas,  and  Minnesota. 


Source  of  Data 
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"7% 
Seattle 
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Demographic  Characteristics  of  Clients 

The  sample  in  the  database  is  composed  of 
clients  of  HTV-case  management  programs.  The 
demographic  profile  of  these  individuals  is 
predominately  male  (82%).  There  are  four  pediatric 
HTV/AIDS  cases  ranging  from  infant  to  four  years  of 
age.  In  the  adolescent/adult  category,  ages  range 
from  16  to  59  years.  The  adolescent/adult  mean  and 
median  age  are  approximately  the  same  at  34  years. 

HIV  transmission  routes  are  recorded  in  the 
database  using  categories  consistent  with  CDC 
surveillance  reporting.  The  pattern  of  HTV  infection 
routes    appears    to    be    similar    to    many    other 
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population  groups.  The  largest  transmission  route 
was  men  having  sex  with  men  comprising  54%  of 
the  sample.  An  additional  8%  contracted  HIV 
through  men  having  sex  with  men  and  IV  drug  use. 
The  sample  has  18%  acquired  HIV  from  IV  drug  use 
alone.  Seventeen  percent  acquired  HIV  through 
heterosexual  contact.  Finally,  1%  contracted  HIV 
from  blood  products  and  2%  contracted  HIV 
through  maternal  transmission. 

HIV  Transmission  Routes 

(number  of  cases) 

140 
120 
100 

80 

60 

40 

20 

0 
Heterosexual  IV  Drug  Use  Blood  Product 

Men  with  Men      Men  w/  Men  &  IV  Drug  Maternal 

as  of  July  1995 


39 

124 

41 

18 

3                4 

The  health  status  of  clients  as  they  enter  into 
case  management  services  (and  the  database)  is  an 
important  baseline  measurement  to  determine 
service  effectiveness  or  HIV  pathogenesis.  Upon 
entry  into  the  data  base,  adolescent/adult  clients 
have  a  mean  CD4  count  of  341  with  a  range  from  0 
to  1,100.  At  the  most  current  measurement  of  CD4 
counts,  the  mean  decreases  to  297  with  a  range 
from  0  to  1,003. 

Case  workers  report  that  47%  of  clients  have 
either  good  or  excellent  health.  Another  35%  have 
fair  health,  while  the  18%  remaining  have  poor 
health. 


Health  Status  of  Clients 

(number  of  cases) 
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Another  measure  of  health  status  has  been 
developed  by  the  CDC  and  is  included  as  a  variable 
in   the   database.      Specifically,   the    1993   revised 


classification  system  for  HIV  infection  has  been 
used  for  comparability  with  other  health  databases. 
This  variable  identifying  the  HIV/AIDS  case  de- 
finition is  illustrated  in  Table  1. 


TABLE  1 .   HIV/AIDS  STATUS 


Count 


Asymptomatic   HIV 

84 

35.9% 

Symptomatic   HIV 

68 

29.1% 

AIDS    Status 

82 

35.0% 

Total 

234 

100.0% 

The  database  also  contains  variables  that 
identify  the  health  and  social  service  programming 
needs  of  individuals.  These  variables  identify  social 
indicators  and  suggest  treatment  needs.  Twenty- 
two  percent  of  clients  in  the  database  have  a  history 
of  mental  illness.  Sixty-two  percent  have  a  history 
of  alcohol  abuse  while  51%  have  a  history  of  drug 
abuse.  Twenty-six  percent  of  clients  have  a  history 
of  being  homeless. 

Important  Preliminary  Findings 

Two  issues  are  particularly  noteworthy  in 
this  data  set.  First,  the  presence  of  American 
Indian,  Alaska  Native,  and  Native  Hawaiian  women 
in  the  database  are  over-represented  when 
compared  with  the  infection  rates  of  women  in  non- 
minority  populations.  However,  the  18%  of  females 
in  the  database  is  approximately  consistent  with 
many  other  minority  populations.  Second,  there  is 
a  low  usage  (48%)  of  the  US  Public  Health  Service, 
Indian  Health  Service  (IHS)  among  the  HIV- 
infected  sample  in  this  database.  This  suggests  that 
despite  important  needs  for  medical  care,  American 
Indians,  Alaska  Natives,  and  Native  Hawaiians  may 
be  reluctant  or  unable  to  access  IHS  services.  This 
finding  is  in  contrast  to  a  popularly  held  belief  that 
"American  Indians  have  their  health  care  needs  met 
by  the  IHS."  This  database  reveals  that  slightly 
more  than  one-half  of  those  needing  HIV  care  were 
unwilling  or  unable  to  access  IHS  services 
supporting  the  concept  that  the  IHS  functions  as 
the  medical-care  provider  of  "last  resort." 

The  first  developmental  phase  of  the  national 
database  is  now  complete  and  still  continuing  to 
progress  with  the  ongoing  collection  of  basic 
demographic  data  as  previously  described.  Over  the 
next  two  years,  analysis  will  be  conducted  of  other 
variables  that  measure  the  longitudinal  aspects  of 
client  status  in  a  case  management  program 
designed  for  American  Indian,  Alaska  Native,  and 
Native  Hawaiian  clients. 
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THE  ONTARIO  HIV  OBSERVATIONAL  DATABASE:  INFORMATION  FOR 
RESEARCH  AND  PLANNING  FOR  PERSONS  LIVING  WITH  HIV  IN  ONTARIO 

Robert  Throop,  Sunnybrook  Health  Science  Center 


Paper  not  available  for  publication. 
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USING  SURVEILLANCE  DATA  IN  COMMUNITY  PLANNING  FOR  HIV  PREVENTION 

Cibele  Barbosa 
Centers  for  Disease  Control  and  Prevention 


Paper  not  available  for  publication, 
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EVALUATION  OF  MALE  EXCESS  AMONG  PRETERM  BIRTHS: 
A  POSSIBLE  ETIOLOGIC  MARKER 

Michael  Cooperstock,  University  of  Missouri  School  of  Medicine 


Paper  not  available  for  publication, 
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RACE,  CLASS,  GENDER  AND  CANCER 

Nancy  Breen,  National  Cancer  Institute 
Alex  Ching 


The  primary  measures  for  assessing  the 
impact  of  cancer  in  the  general  population 
are  the  age-adjusted  rates  of  incidence 
(number  of  new  cases  per  year  per  100,000 
persons)  and  mortality  (number  of  deaths 
per  100,000  persons).  Mortality  from 
cancer  is  higher  for  black  than  white 
Americans.  The  cancer  mortality  rate  for 
white  men  between  1986-90  was  213.2,  and  for 
black  men  it  was  315.  For  white  and  black 
women  it  was  139.2  and  165.7  respectively. 
Incidence  rates  for  the  same  period  were 
450.4  and  540  for  white  and  black  men  and 
345.8  and  328.8  for  white  and  black 
women.  (1)  Higher  mortality  for  black  women 
is  especially  troubling  because  incidence 
is  lower  for  black  than  white  women. 

Why  is  mortality  from  cancer  higher  for 
black  than  white  Americans?  The  literature 
has  suggested  a  number  of  reasons: 

—socioeconomic  status 

—late  stage  of  disease  at  diagnosis 

—diagnostic  and  treatment  delays 

—treatment  differences 

—biological  and  constitutional  factors 

—environmental  factors. 


context  of  health  care  delivery  for  cancer. 
We  will  then  evaluate  literature  and 
suggest  refinements  to  data  collection  or 
analytical  methods. 

For  breast  and  cervical  cancer,  a 
person  diagnosed  with  cancer  is  more  likely 
to  survive  if  the  cancer  is  detected  early 
and  treated  promptly .  However,  white  women 
diagnosed  with  breast  or  cervical  cancer 
tend  to  survive  longer  than  black  women.  A 
key  indicator  of  cancer  prognosis  is  stage 
at  the  time  of  diagnosis. 

There  are  four  stages  at  which  cancer 
may  be  diagnosed.  These  include  1)  in  situ: 
a  neoplasm  with  all  characteristics  of 
malignancy  except  invasion  (data  not  pub- 
lished by  NCI),  2)  localized:  an  invasive 
malignant  neoplasm  confined  entirely  to  the 
organ  of  origin,  3)  Regional:  an  invasive 
neoplasm  which  a)  extends  beyond  the  organ 
of  origin  directly  into  surrounding  tissue 
or  b)  involves  regional  lymph  nodes  or  c) 
both,  4)  Distant:  an  invasive  neoplasm 
which  has  spread  to  part  of  the  body  remote 
from  the  primary  tumor  site. 


This  research  addresses  the  question  to 
what  extent  are  race  differences  in  cancer 
mortality  attributable  to  SES  differences? 
One  study  found  race  differences  in  mortal- 
ity for  all  cancer  sites  aggregated  are 
entirely  explained  by  age,  sex,  marital 
status,  family  size  and  income. (2)  Because 
cancer  is  a  family  of  related  diseases, 
rather  than  a  single  disease,  cancer  is 
usually  analyzed  by  particular  site. 

This  paper  examines  the  health  care 
delivery  system  and  tests  a  two-part  socio- 
economic hypothesis.  First,  mechanisms 
related  to  economic  inequality  perpetuate 
pre-existing  racism  in  the  health  care 
delivery  process.  Second,  this  process  is 
cumulative . 

Our  research  plan  is  to  examine  two  case 
studies,  breast  and  cervical  cancer,  for 
which  screening  has  a  proven  mortality 
benefit.  Breast  cancer  was  the  most  common 
cancer  among  women  in  1990.  (3)  We  will 
examine  the  extent  to  which  the  literature 
has  analyzed  race  and  SES  factors  in  the 


Table  1  shows  5-yr  survival  by  stage  for 
breast  and  cervical  cancer  diagnosed  be- 
tween 1983-87  and  the  percent  of  black  and 
white  women  diagnosed  at  each  stage. (1) 

Table  1 


whi 

t  e 

Women 

Black 

Women 

Percent 

Percent 

Percent 

Percent 

survivi 

ig 

diagnosed 

surviving 

diagnosed 

BREAST 

CANCER 

Local 

94 

53 

84 

42 

Regional 

73 

36 

57 

42 

Distant 

19 

7 

10 

10 

CERVICAL    CANCER 

Local 

90 

51 

86 

36 

Regional 

53 

32 

45 

41 

Distant 

13 

10 

14 

14 

The  combination  of  the  result  of 
differentials  in  survival  and  the  percent 
black  and  white  women  diagnosed  at  each 
stage  of  cancer  results  in  mortality  rates 
of  22.4  and  26.1  for  breast  cancer  and  2.3 
and  6  for  cervical  cancer  for  white  and 
black  women  respectively.  Later  stage 
breast  and  cervical  cancer  diagnosis  ap- 
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pears  to  be  part  of  the  explanation  for 
higher  mortality  from  breast  cancer  for 
black  than  white  women  and  SES  is  associated 
with  stage  of  diagnosis. 


Census-tract  level  SES  (17)  or  private 
health  insurance  status. (18)  Cervical 
cancer  stage  differences  are  associated 
with  both  race  and  SES  (19, 20) 


Studies  have  shown  that  SES  explains 
some  or  all  of  race  differences  in  breast 
or  cervical  cancer  survival.  For  example, 
a  recent  study  by  Eley  using  data  from  the 
SEER  special  black/white  study  found  the 
mortality  risk  of  breast  cancer  was  2.1 
times  greater  for  black  than  white  women. 
Forty  percent  of  this  difference  was  due  to 
later  stage  at  diagnosis  for  black  women. 
The  authors  concluded  that  sociodemographic 
variables  appear  to  act  largely  through 
racial  differences  in  stage  at  diagno- 
sis. (4)  Other  studies  found  that  while  race 
was  a  significant  predictor  of  cancer 
survival,  when  an  indicator  of  SES  was 
included  in  the  model,  race  ceased  to  be 
significant  or  was  attenuated.  (5-12 ) 

Standard  medical  care  for  breast  or 
cervical  cancer  includes  screening  (for  the 
entire  population)  ,  diagnosis  that  in- 
cludes staging,  treatment,  and  post-treat- 
ment care  and  monitoring.  Literature  on 
access  to  care  at  each  point  during  course 
of  standard  treatment  for  race  and  SES 
differences  will  be  examined. 

ACCESS  TO  STANDARD  CARE 
Screening 

Screening  modalities  for  breast  cancer 
include  mammography,  breast  physical  ex- 
amination (performed  by  a  health  care 
provider)  and  breast  self-examination. 
Mammography  has  been  shown  to  reduce  breast 
cancer  mortality  by  approximately  thirty 
percent  in  clinical  trials  (13)  and  is 
therefore  generally  considered  the  most 
reliable  breast  cancer  screening  modality. 
Population  studies  have  shown  the  Pap  smear 
to  be  effective  for  cervical  cancer;  if 
women  were  regularly  screened,  it  could 
lead  to  a  near  eradication  of  cervical 
cancer . 

Rates  of  utilization  of  screening  are 
monitored  in  the  National  Health  Interview 
Survey.  Recent  studies  show  that  SES 
accounted  for  differences  by  race  in 
mammography  use.  (14,15)  However,  it  did 
not  account  for  differences  in  use  of  Pap 
smears  by  race  or  Hispanic  ethnicity.  (16) 


Treatment 

For  breast  cancer,  Bain (21)  found  no 
racial  difference  in  the  percent  of  women 
with  local  or  regional  disease  who  received 
surgery;  for  women  with  distant  disease, 
there  was  a  racial  difference.  Black  women 
with  advanced  disease  were  less  likely  than 
white  women  to  receive  surgery  and  their 
mortality  was  higher  than  white  women  with 
advanced  disease. 

SUMMARY 

SES  appears  to  underlie  racial  differ- 
ences at  each  stage  of  cancer  care. 
However,  it  is  difficult  to  draw  firm 
conclusions  from  the  literature  because 
many  studies  examine  only  race  or  only  SES 
(or  neither)  .  To  draw  firm  conclusions, 
additional  studies  that  examine  race  and 
SES  simultaneously  are  needed. 

Consequently,  we  cannot  systematically 
evaluate  our  hypothesis.  Though  more  than 
sixty  articles  were  reviewed,  and  the 
articles  cited  suggest  the  importance  of 
social  class  in  obtaining  prompt  access  to 
standard  care  for  breast  and  cervical 
cancer,  the  literature  does  not  routinely 
examine  race  and  social  class  for  both 
cancers  at  each  point  in  the  process  of 
cancer  care  so  the  hypothesis  can  not  be 
evaluated.  We  found  no  studies  on  treatment 
for  cervical  cancer  or  post-treatment  care 
and  follow  up  for  either  cancer  that 
analyzed  race  and  SES. 

Except  for  NHIS  data  on  screening,  few 
cancer  sources  collect  family  size,  income, 
education,  health  insurance  status  or  other 
variables  related  to  SES  or  social  class. 

RECOMMENDATIONS 

SES  variables  need  to  be  routinely 
collected  on  health  surveys.  This  data  can 
be  collected  from  respondents.  Where 
medical  abstracts  are  the  source  of  infor- 
mation, detailed  insurance  information  can 
be  collected  as  a  rough  proxy  for  SES.  Geo- 
coding  at  the  Census  tract  level  also 
provides  a  useful  proxy  for  SES  data  when 
individual  data  are  not  available. 


Diagnosis 

Racial  differences  in  breast  cancer 
stage   at   diagnosis   were   explained  by 


SES  needs  to  be  explicitly  compared 
with  race.  Public  health  studies  often  use 
race  as  a  proxy  for  social  class.   While  a 
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far  greater  proportion  of  blacks  and 
Hispanics  than  whites  are  poor,  there  are 
social  class  differences  by  race  and 
ethnicity.  These  two  variables  need  to  be 
routinely  analyzed  separately  and  explic- 
itly in  order  to  sort  out  the  effects  of  race 
and  the  effects  of  social  class. 

SES  needs  to  be  tested  as  an  explanatory 
variable  because  it  is  subject  to  policy 
intervention.  Epidemiological  studies 
often  use  SES  as  a  control  variable.  Social 
class  (and  income  distribution)  are  social 
phenomena  that  can  be  altered  by  public 
policy  interventions.  Public  policy  is 
more  likely  to  succeed  if  it  is  based  on 
scientific  evidence,  however,  and  cur- 
rently we  do  not  have  the  evidence  we  need 
to  design  sound  public  policy. 
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EXAMINING  THE  BLACK-WHITE  ADULT  MORTALITY  DISPARITY:  THE  ROLE  OF  RESIDENTIAL  SEGREGATION 
Chiquita  Collins  and  David  R.  Williams,  University  of  Michigan 


During  the  past  several  decades  the  United 
States  has  experienced  substantial  improvements 
in  the  health  of  all  of  its  citizens  but 
African  Americans'  continue  to  experience 
higher  rates  of  mortality  than  whites.  Table  1 
indicates  how  pervasive  racial  disparities  in 
mortality  are.  It  presents  the  overall  death 
rates  for  blacks  and  whites,  and  the  black- 
white  ratios  for  the  15  leading  causes  of  death 
in  the  United  States  in  1990.  A  ratio  greater 
than  1  indicates  that  the  death  rate  for  blacks 
is  higher  than  for  whites.  With  the  exception 
of  suicides  and  chronic  obstructive  pulmonary 
diseases,  mortality  rates  from  all  causes  of 
death  were  higher  for  blacks  than  for  whites. 
The  differences  are  small  for  some  conditions 
such  as  atherosclerosis  and  accidents,  but 
substantial  for  others,  such  as  kidney  disease, 
perinatal  conditions,  AIDS  and  homicide. 

Table  1.   Age-Adjusted  Death  Rates  for  the 

15  Leading  Causes  of  Death 

By  Race  in  the  United  States,  1990 

(per  100,000  Population) 


B-W 

Cau 

se  of  Death 

Blacks 

Whites 

Ratio 

1. 

Heart  Disease 

213.5 

146.9 

1.45 

2. 

Cancer 

182.0 

131.5 

1.38 

3. 

Stroke 

48.4 

25.5 

1.90 

4. 

Accidents 

39.7 

31.8 

1.25 

5. 

Pulmonary 

16.9 

20.1 

0.84 

6. 

Pneumonia  and 

19.8 

13.4 

1.48 

7. 

Diabetes 

24.8 

10.4 

2.38 

8. 

Suicide 

7.0 

12.2 

0.57 

9. 

Liver  Disease 

13.7 

8.0 

1.71 

10. 

HIV-AIDS 

30.6 

9.8 

3.12 

11. 

Homicide 

39.5 

5.9 

6.69 

12. 

Kidney  Disease 

10.8 

3.6 

3.00 

13. 

Septicemia 

9.5 

3.5 

2.71 

14. 

Atherosclerosis 

3.1 

2.7 

1.15 

15. 

Perinatal 

16.2 

5.2 

3.12 

Prior  research  has  provided  important 
evidence  about  the  existence  and  magnitude  of 
the  black-white  mortality  gap,  but  how  and  why 
this  disparity  persists  is  not  well  understood. 
Differences  in  social  and  economic  status 
(Adler,  et  al .  1993;  Pappas  et  al.  1993), 
accessibility  and  quality  of  medical  care 
(Blendon,  et  al.  1989),  working  conditions, 
environmental  factors,  as  well  as  genetic  and 
life  style  factors  (Otten,  et  al.  1990)  have 
been  proposed  as  explanations  of  the  mortality 
gap  between  blacks  and  whites  (Williams  and 
Collins,  1995) .  The  empirical  evidence  to  date 
suggests  that  these  factors  have  important  but 
limited  explanatory  value  in  accounting  for 
racial  differentials  in  mortality.  There  is 
growing  interest  in  identifying  the  ways  in 
which  racism  is  causally  prior  to  these 
traditional  factors  and  operates  additively  and 
interactively  with  them  to  influence  the 
observed  rates  of  disease  (Krieger  et  al.  1993; 
Cooper  1993;  Williams  et  al.  1994).  A 
central  aspect  of  racism  is  residential 
segregation,  the  extent  to  which  blacks  live 
separately  from  whites  (Massey  and  Denton, 
1993) . 

The    literature    proposes 
mechanisms    by    which    racial 
segregation  may  affect  the  lives  of  African 
Americans   (Massey   and   Denton,   1993,   Wilson 


two    major 
residential 


1987) .  First,  it  creates  and  perpetuates 
spatially  concentrated  poverty  (Massey  and 
Denton,  1993) .  Second,  it  socially  isolates 
African  Americans  from  white  society  (Wilson, 
1987).  Both  mechanisms  can  adversely  affect 
health. 

Because  of  the  concentration  of  poverty, 
poor  African  Americans  may  be  different  to  the 
poor  of  other  groups.  In  many  urban  areas 
there  is  a  high  degree  of  segregation  combined 
with  a  high  rate  of  poverty,  and  low  levels  of 
education  and  job  skills  (Massey  et  al.  1991). 
Living  in  poor  segregated  neighborhoods  can 
include  dilapidated  housing,  poor  nutrition, 
unemployment  and  underemployment,  low-wage 
jobs,  poor  education,  poor  police  protection, 
crime,  high  cost  of  living,  fatal  injuries,  and 
a  lack  of  health  insurance  and  access  to  basic 
health  services.  African  Americans  living  in 
segregated  neighborhoods  are  exposed  to  these 
conditions  more  frequently  than  their  white 
counterparts.  On  average,  poor  African 
Americans  are  increasingly  concentrated  in 
neighborhoods  with  bad  living  conditions,  while 
poor  whites  are  more  evenly  dispersed 
throughout  the  city,  with  many  residing  in 
relatively  safe  and  comfortable  neighborhoods 
(Wilson  1987) .  The  risk  of  teenage  pregnancy 
and  early  sexual  intercourse  is  high  in  poor 
neighborhoods  (Hogan  and  Kitagawa  1985) .  Crane 
(1991)  found  that  as  the  percentage  of  low-SES 
workers  in  a  poor  black  neighborhood  rose  there 
was  an  increasing  likelihood  of  dropping  out  of 
high  school  and  having  a  teen  birth. 
Similarly,  Massey  et  al.  (1991)  report  that 
increases  in  neighborhood  poverty  rates  are 
associated  with  increases  in  male  joblessness 
and  the  percentage  of  female-headed  households. 

Residential  segregation  has  an  important 
role  in  perpetuating  social  problems  in 
predominantly  black  neighborhoods  because  the 
degree  of  isolation  from  mainstream  society 
creates  a  neighborhood  environment  among 
African  Americans  that  is  substandard  in 
quality  when  compared  to  whites.  African 
Americans  living  in  highly  segregated  areas  are 
unlikely  to  have  a  protective  barrier  to  buffer 
them  from  exposure  to  adverse  living  conditions 
such  as  drug  trafficking  and  illicit  drug 
abuse,  high  rates  of  crime,  poor  quality 
schools,  alcoholism,  disproportionate  number  of 
liquor  stores,  dilapidated  housing,  and  vacant 
lots  used  as  garbage  dumps.  Table  2  shows 
characteristics  that  are  commonly  found  in 
predominantly  poor,  black  neighborhoods  and 
potential  health  risk  factors  as  a  result  of 
racial  residential  segregation. 

On  average  there  are  vast  disparities 
between  blacks  and  whites  in  the  social  and 
economic  composition  of  their  neighborhoods. 
Compared  to  white  neighborhoods,  African 
American  neighborhoods  are  more  likely  to  have 
numerous  fast  food  restaurants,  auto  mechanic 
shops,  liquor  stores,  beauty  shops  and  check 
cashing  centers.  Goods  and  services  provided 
in  black  neighborhoods  are  on  average  worse  in 
quality  than  those  in  white  neighborhoods.  For 
example,  since  supermarkets  are  located  outside 
most  black  neighborhoods,  many  blacks  are 
limited  to  shopping  for  food  at  local  groceries 
and  corner  stores  where  food  is  more  expensive 
but  lower  in  quality  than  in  suburban 
supermarkets.  Blacks  often  pay  15%  more  for 
food  and  22%  more  for  vegetables  in  black 
neighborhoods   (Troutt   1993)  .    Wilson   (1987) 
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Table  2.   Neighborhood  characteristics  that  are  common  in  predominantly  poor,  black 
neighborhoods  and  potential  risk  factors/health  outcomes  that  are  affected  by  such 
neighborhoods  as  a  result  of  residential  segregation. 


Neighborhood  Characteristics 

Health  Outcomes 

Housing 

Dilapidated  housing,  public  housing 
Boarded-up  housing,  poor  plumbing,  poor 
heating,  poor  ventilation,  poor  maintenance 

Tuberculosis,  Asthma,  Chronic 
Bronchitis,  Accidents 

Employment 

low  skill/training,  low  education  attainment, 
access  to  jobs  (suburbanization  of  low  skill 
high  wage  jobs) 

Cirrhosis  of  the  liver  and 
chronic  liver  disease 
(alcoholism) ,  Occupational 
Accidents,  Exposure  to  Toxins 

Education 

ratio  of  students  to  teacher,  overcrowded 
classes,  poor  student  performance,  high  drop- 
out rates,  "tracking" 

Cirrhosis  of  the  liver  and 
chronic  liver  disease 
(alcoholism) ,  illicit  drug 
use,  cigarette  smoking 

Disproportionate  #  of  liquor  stores 
marketing  and  selling  strategies  to  sell  hard- 
liquors  and  malt  liquors,  (beer  with  high 
alcohol  content) 

Cirrhosis  of  the  liver  and 
chronic  liver  disease 

Police  Protection/Safety 

high  rates  of  victimization  of  violent  crimes 

(e.g.  assault  w/weapon,  homicide,  robbery) 

Homicide,  accidents 

Informal  support  mechanisms 

<,   2  friends 

high  levels  of  social  isolation 

high  rates  of  single-female  headed  households 

Suicide,  Cirrhosis  of  the 
liver  and  chronic  liver 
disease  (alcoholism),  illicit 
drug  use 

Quality  of  food 

higher  cost,  poorer  quality  (e.g.  spoiled 

meat,  welted  vegetables) 

Hypertensive  disease, 
Diabetes,  Cancer,  Obesity 

Cost  of  living 

high  cost  for  rent,  high  taxes,  fixed  incomes 

Lack  of  resources  for  adequate 
goods  and  services 

contends  that social isolation  produced  by the 
absence  of  social  interaction  between  African 
Americans  and  white  Americans  adversely  impacts 
the  living  conditions  in  highly  concentrated 
poor  black  neighborhoods. 

According  to  this  view  limited  exposure  to 
white  society  (a  result  of  residential 
segregation)  ensures  that  poor  African 
Americans  lack  direct  linkages  to  quality 
services  from  societal  institutions.  For 
example,  because  residence  is  associated  with 
educational  opportunities,  racial  residential 
segregation  perpetuates  an  educational 
disadvantage  for  poor  blacks  by  exposing  them 
to  inferior  schools  (Coleman,  Kelley  &  Moore 
1975,  Farley  1978,  Farley  &  Taueber  1975) .  In 
urban  schools,  African  American  students  are 
more  likely  to  be  placed  (or  "tracked")  in  low 
-ability  and  non-college  preparatory  programs. 
Black  students  that  are  tracked  are  placed  in 
classrooms  where  the  teachers'  low  expectations 
contribute  to  lower  levels  of  school 
performance  and  attainment  (Jaynes  &  Williams 
1989) .  In  addition  to  tracking,  blacks  are 
twice  as  likely  as  whites  not  to  graduate  from 
high  school  (Jaynes  &  Williams  1989) .  Inferior 
schools  in  combination  with  low  educational 
aspirations  limit  African  Americans'  access  to 
good  employment  opportunities  and  other  formal 
networks  and  institutions  thereby  creating  a 
disadvantage  in  competing  for  employment 
opportunities  with  desirable  salaries  and 
working  conditions.  The  result  is  that 
employed  African  Americans  are 
disproportionately  found  in  low-skilled  and 
low-paid  occupational  positions.  In  short, 
African  American  neighborhoods  that  are 
socially  isolated  from  mainstream  society  are 
disproportionately  affected  by  social  and 
economic  problems,  that  occur  less  frequently 
in  predominantly  white  neighborhoods.  These 
include  low-achieving  schools,  joblessness,  low 
rates  of  marriage,  and  crime  which  in  turn, 
have  deleterious  effects  on  their  quality  of 
life. 


Studies  that  have  examined  the  association 
between  segregation  and  health  have 
consistently  found  segregation  to  be  a 
predictor  of  variation  in  mortality.  Yakauer 
(1950)  reported  that  infant  mortality  rates  for 
blacks  and  whites  were  highest  in  the  most 
segregated  areas  (all-black  areas).  More 
recent  studies  have  found  that  residential 
segregation  was  significantly  related  to  health 
status  for  blacks  but  not  for  whites  for  the 
following  death  rates:  infant  mortality 
(LaVeist  1989;  Polednak  1991),  all-cause 
mortality  for  persons  aged  15-24  and  65-74 
(Polednak  1993),  and  homicide  (Rosenfield  1986; 
Potter  1991).  LaVeist  (1989)  found  a  positive 
association  between  residential  segregation  and 
black  infant  mortality,  but  an  inverse 
association  with  white  infant  mortality.  In  a 
similar  study,  Polednak  (1991)  found  that  the 
most  important  predictor  of  the  black-white 
difference  in  infant  mortality  in  38  standard 
metropolitan  statistical  areas  (SMSAs)  was  an 
index  of  residential  dissimilarity,  independent 
of  the  difference  in  median  family  income  and 
the  prevalence  of  poverty. 

Logan  and  Messner  (1987)  found  a 
significant  relationship  between  residential 
segregation  and  homicide  in  suburban  rings  of 
54  metropolitan  areas  in  1980  and  not  in  1970. 
In  contrast,  Rosenfield  (1986)  found  that 
residential  segregation  had  a  positive 
association  with  homicide  in  standard 
metropolitan  statistical  areas  (SMSAs)  for 
1970.  Potter  reported  that  the  level  of 
residential  isolation  had  a  significant  effect 
on  the  homicide  differential  between  blacks  and 
whites  in  1980  for  27  metropolitan  areas.  In  a 
recent  study,  Polednak  (1993)  examined  the 
association  between  residential  segregation  and 
age-specific  all-cause  mortality  rates  and 
found  that  the  level  of  residential  segregation 
was  a  positive  predictor  of  the  black-white 
ratio  for  persons  between  the  ages  of  15  and 
44.   In  a  study  that  only  analyzed  the  effects 
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of  racial  residential  segregation  on  black 
homicide,  Peterson  and  Krivo  (1993)  found  that 
racial  residential  segregation  had  a  strong 
significant  positive  effect  on  black  homicide 
rates  for  central  cities  of  SMSAs  in  1980. 

In  this  paper  we  examine  the  extent  to 
which  residential  segregation  can  account  for 
the  black-white  adult  mortality  disparity. 
Unlike  previous  studies  of  all-cause  mortality, 
we  used  cities  as  the  unit  of  analysis  instead 
of  standard  metropolitan  statistical  areas 
(SMSAs)  which  increased  our  sample  size 
considerably  when  compared  to  previous  studies. 

Data  and  Methods 
Study  Population 

This  study  used  data  from  the  Mortality 
Detail  Files,  collected  by  the  National  Center 
for  Health  Statistics  (NCHS),  and  from  the  1990 
United  States  Census  to  analyze  the 
relationship  between  residential  segregation 
and  the  black-white  mortality  gap.  Our  unit  of 
analysis  is  U.S.  cities  with  a  population  of  at 
least  100,000  in  1990  and  with  an  African 
American  population  of  at  least  ten  percent. 
These  criteria  yielded  a  sample  size  of  107 
cities.  Only  mortality  for  blacks  and  whites 
who  were  at  least  fifteen  years  of  age  at  time 
of  death  were  analyzed.  Mortality  rates  were 
disaggregated  into  three  age  groups:  15  to  44 
years  of  age;  45  to  64  years  of  age;  and  65 
years  and  older.  These  rates  were  then 
adjusted  for  the  age  distribution  of  the  1990 
U.S.  population  using  "direct"  standardization 
techniques  to  obtain  race-  and  gender-  specific 
age  standardized  mortality  rates.  All  race 
specific  information  obtained  from  the  Census 
correspond  with  the  racial  categories  of  the 
mortality  data.  Race  was  defined  as  either 
"black"  or  "white",  excluding  ethnic 
distinctions  (i.e.,  non-Hispanic). 

Measures 
Dependent  Variable 

The  dependent  variable  is  the  difference 
between  the  city's  black  and  white  adult 
mortality  rate  per  100,000  population.  The 
black-white  mortality  disparity  will  be  less 
than  zero  if  the  white  mortality  rate  is 
greater  than  the  black  mortality  rate,  and 
greater  than  zero  if  the  black  rate  is  higher. 


Independent  Variables 

Six  independent  variables  were  selected  as 
determinants  of  the  black-white  mortality  gap: 
one  measure  of  residential  segregation,  four 
measures  of socioeconomic  status,  and  population 
size.  Table  3  lists  the  independent  variables 
and  describes  how  they  were  operationalized. 

Racial  residential  segregation  was  measured 
by  calculating  the  index  of  dissimilarity  for 
each    city     (Massey    & 
Theoretically,     levels 
segregation  between  blacks  and 
from  0  to  100.   It  takes  on  the 


of  100  in  a  situation  where  the  racial 
composition  is  completely  segregated.  A  value 
of  zero  indicates  that  blacks  and  whites  are 
randomly  distributed  across  all  census  blocks 
in  a  given  city.  The  measures  of  socioeconomic 
status  and  population  size  are  straightforward. 
These  variables  are  similar  to  those  used  in 
earlier  studies  of  residential  segregation  and 
mortality.  Education  is  measured  as  the  black- 
white  difference  in  the  proportion  of  persons 
older  than  24  years  of  age  that  graduated  from 
high  school.  Occupational  status  is  measured 
by  obtaining  the  black-white  difference  in  the 
proportion  of  persons  older  than  15  years  of 
age  employed  in  professional  and  managerial 
occupations.  Income  is  measured  by  obtaining 
the  black-white  difference  in  median  family 
income  in  1989  dollars.  Because  the  total 
black  and  white  population  in  cities  varied 
considerably  in  size,  in  all  of  our  analyses  we 
controlled  for  population  size  by  using  the 
natural  log  of  the  total  city  population. 
Statistical  Analysis 

This  report  used  ordinary  least  squares 
(OLS)  analyses  to  assess  the  relationship 
between  racial  residential  segregation  and  the 
black-white  all-cause  mortality  gap.  All 
analyses  are  reported  separately  for  men  and 
women  for  the  three  age  categories  described 
earlier. 
Results 

The  means,  medians,  standard  deviations, 
minimum  and  maximum  values  for  each  variable 
are  shown  in  table  4.  The  index  of 
dissimilarity  in  1990  varied  by  a  factor  of 
almost  three,  ranging  from  31.1  (Aurora, 
Colorado)  to  88.3  (Cleveland,  Ohio).  On 
average,   the  black-white  gap   in  education, 


income,  and  occupational  status  was  greater  in 
highly  segregated  cities  than  in  moderate  to 
low  segregated  cities. 

For  both  men  and  women,  the  black-white 
mortality  gap  varied  considerably  across  the 
107  U.S.  cities.  The  average  age  specific 
mortality  rate  (per  100,000  population)  was 
approximately  twice  as  great  for  blacks  as  for 
whites  between  the  ages  of  15-44  and  45-64;  the 
mortality  gap  was  smaller  for  individuals  aged 
65  and  over.  Durham,  North  Carolina  had  the 
largest  black-white  mortality  gap  for  women 
aged  15-44,  and  a  moderately  high  segregation 
index  (60.0),  while  the  largest  mortality  gap 
among  men  was  evident  in  New  Haven,  Connecticut 
with  a  slightly  higher  segregation  index 
(62.2).  For  men  and  women  between  the  ages  of 
45-64,  the  largest  black-white  mortality  gap 
was  present  in  Berkeley,  California  and 
Virginia  Beach,  Virginia  respectively. 
Denton  1987)  . 
of  residential 
whites  range 
maximum  value 

Table  3.   Independent  Variables  for  the  Analysis  of  the 
Black-White  Racial  Gap  in  Mortality 


Variables 

Ope rationalization 

Segregation 

Dissimilarity 

SES 

B-W  income  gap 

B-W  education  gap 
B-W  occupation  gap 
Population 

Index  of  dissimilarity  of  black-white 
residential  segregation  across  census  blocks 

Difference  of  black-white  median  family  income 
(in  1989  dollars) 

Difference  of  black-white  population  older 
than  24  with  at  least  a  high  school  education 
Difference  of  black-white  employed  persons  in 
professional  and  managerial  occupations 
Natural  log  of  total  black  and  white 
population 
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Table  4.  Means,  Standard  Deviations,  and  Ranges  for  Residential  Segregation  and 
Socioeconomic  Status  Indicators  by  Race: 
U.S.  Cities  (100,000  population  or  more) 


Segregation 

Education 

Income 

Occupation 

Poverty 

(%) 

(1989  dollars) 

(%) 

(%) 

Median 

65.9 

Black  White 

Black    White 

Black  White 

Black   White 

64.4 

79.9 

$21,028 

$36,691 

16.5 

30.2 

28.  1 

6.7 

Mean 

65.4 

64.7 

77.9 

$21,219 

$37,837 

17.5 

31.6 

28.3 

7.6 

Standard 

Deviation 

11.2 

6.86 

8.99 

$4,397 

$7,869 

3.5 

8.3 

7.1 

3.6 

Range: 

Minimum 

31.1 

39.7 

42.5 

$13,883 

$22,050 

11.0 

13.2 

10.6 

1.9 

Maximum 

88.3 

87.3 

96.3 

$33,410 

$76,074 

29.5 

62.0 

43.6 

19.6 

Atrican  Americans  living  in  highly 
segregated  cities,  on  average,  were  less  likely 
to  graduate  from  high  school,  earn  high 
incomes,  and  hold  managerial  and  professional 
occupations  than  blacks  living  in  least 
segregated  cities.  The  pattern  was  similar  for 
whites  for  education,  but  whites  living  in 
least  segregated  cities  had  slightly  higher 
levels  of  income  and  occupational  status  than 
their  counterparts  in  highly  segregated  cities. 

Table  5  presents  the  relationship  between 
residential  segregation  and  the  black-white 
mortality  gap  for  women.  Residential 
segregation  has  a  marginally  significant 
positive  association  with  the  black-white 
mortality  gap  for  women  aged  15-44.  However, 
this  relationship  was  non-significant  for  women 
between  the  ages  of  45-64  and  65  and  older. 
The  pattern  for  men  under  the  age  of  65  was 
similar  to  that  of  women  (Table  6) .  The 
residential  segregation  index  was  positively 
related  to  the  black-white  mortality  gap  for 
men  aged  15-44  (p<.01)  but  unrelated  for  those 
in  the  45-64  age  category.  Thus,  for  both 
males  and  females  in  young  adulthood  and  early 
middle  age,  the  black-white  gap  in  mortality 
tends  to  be  greater  in  cities  that  have  high 
levels  of  residential  segregation.  In 
contrast,  it  is  striking  that  although  the 
association  between  segregation  and  the  black- 
white  mortality  gap  is  significant  for  men  aged 
65-85+,  the  direction  of  the  association  is 
negative. 

Turning  to  the  socioeconomic  variables  used 
in  the  analyses,  both  education  and  occupation 
were  associated  with  the  black-white  mortality 
gap  for  women  aged  15-44,  while  occupation  was 
associated  with  the  mortality  gap  for  women 
between  the  ages  of  45-64.  When  the  analysis 
is  performed  for  women  aged  65  and  over  neither 


education,  income,  nor  occupation  were  related 
to  the  black-white  mortality  gap.  These 
findings  suggest  that  among  females,  education 
and  occupation  played  a  role  in  reducing  the 
black-white  mortality  gap,  but  only  at  the 
younger  ages.  Among  males,  none  of  the  SES 
indicators  played  an  important  role  in  reducing 
the  black-white  mortality  gap.  There  is  one 
exception:  education  was  associated  with  the 
black-white  mortality  gap  for  men  over  the  age 

of   64.     Thus,   the   socioeconomic   factors 
analyzed  in  this  study  played  only  a  modest 
role   in   explaining   racial   differences   in 
mortality. 
Discussion 

Our  analyses  reveal  that  residential 
segregation  is  only  modestly  related  to  black- 
white  differences  in  mortality,  and  that  this 
pattern  is  most  evident  for  males  and  females 
under  age  45.  Several  limitations  of  the 
present  study  should  be  mentioned.  First, 
mortality  rates  were  not  adjusted  for  census 
undercount  which  tends  to  bias  upward  the  rates 
for  African  American  males.  Demographers  have 
long  indicated  that  the  coverage  of  African 
American  males  in  young  adult  and  middle-age 
groups  in  U.S.  decennial  censuses  is  less 
complete  than  that  of  white  Americans  (Notes  & 
Comments,  1994)  . 

A  second  problem  with  these  data  is 
potential  error  linked  to  racial 
misclassification.  In  most  cases,  racial 
information  in  the  census  is  obtained  from  one 
household  member  who  provides  information  for 
not  only  himself,  but  for  all  household 
members.  More  importantly,  racial  information 
in  the  mortality  records  comes  from  death 
certificates.  Funeral  directors  typically 
complete  this  information  based  on  their  own 
judgment.   Massey  (1980)  found  that  6%  of  self- 


Table  5.  Unstandardized  Regression  Coefficients  for  the  Association  of  Segregation, 
Population  Size  and  SES  to  the  Black-White  All-Cause  Mortality  Gap:  Women 


(15- 

-44) 

(45- 

-64) 

(65- 

85+) 

b 

S.E. 

b 

S.E. 

b 

S.E. 

Segregation 

1.359* 

(.619) 

.271 

2.161 

-1.580 

12.553 

Pop.  Size 

-11.304 

(8.807) 

10.420 

30.725 

-84.410 

178.515 

Education  Gap 

3.583** 

(1.220) 

1.913 

4.257 

21.781 

24.735 

Income  Gap 

-6.96E-04 

(.002) 

.003 

.006 

-.038 

.033 

Occupation  Gap 

-2.973+ 

(1.756) 

-13.401+ 

6.127 

-38.685 

35.600 

Constant 

136.772 

96.992 

869.725 

R2 

.103 

.097 

.091 

N  =  107 

+=p<.10;  *=p<.05;  **=p<.01 
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Table  6.  Unstandardized  Regression  Coefficients  for  the  Association  of  Segregation, 
Population  Size  and  SES  to  the  Black-White  All-Cause  Mortality  Gap:  Men 


(15- 

-44) 

(45- 

-64) 

(65- 

85+J 

b 

S.E. 

b 

S.E. 

b 

S.E. 

Segregation 

4.513** 

1.455 

5.800 

4.614 

-48.460** 

18.063 

Pop.  Size 

-13.044 

20.693 

-72.898 

65.609 

274.207 

256.869 

Education  Gap 

3.701 

2.867 

-2.017 

9.091 

-82.182* 

35.592 

Income  Gap 

6.32E-04 

.004 

.002 

.012 

.034 

.048 

Occupation  Gap 

-5.443 

4.127 

-12.239 

13.084 

-38.836 

51.226 

Constant 

92.366 

1055.802 

-341.551 

R2 

.111 

.055 

.162 

N  =  107 

+=p<.10;  *=p<.05;  **=p<.01 

identified  African  Americans  were  misclassif ied 
by  their  interviewer  as  white.  Third,  our 
aggregate  analyses  are  subject  to  the 
"ecological  fallacy"  and  our  findings  do  not 
directly  speak  to  the  nature  of  the  association 
between  racism  and  health  at  the  individual 
level . 

The  present  analyses  show  an  association 
between  racial  residential  segregation  and  the 
black-white  mortality  disparity  for  ages  15-44 
for  men  and  women,  even  after  SES  factors  are 
taken  into  account.  This  association  between 
segregation  and  mortality  could  reflect  the 
limited  access  to  health  enhancing  resources 
that  promote  good  health.  Residential 
segregation  may  expose  African  Americans  to  a 
broad  range  of  risk  factors  that  increases 
their  vulnerability  to  diseases.  As  a  result 
of  this  exposure  to  bad  living  conditions,  many 
African  Americans  may  experience  stress  which 
in  turn  can  affect  physical  and  mental  health. 
However,  the  underlying  mechanisms  and 
processes  need  to  be  identified  and  empirically 
verified. 

Our  findings  here,  though  modest,  point  to 
the  importance  of  understanding  neighborhood 
effects  on  health  status.  One  study  examined 
the  joint  effects  of  income,  social  isolation, 
and  depression  on  the  risk  of  mortality  using 
data  from  the  Alameda  County  Study. 
Researchers  found  that  individuals  who  were 
poor,  socially  isolated,  and  depressed  were 
four  times  as  likely  to  die  than  compared  to 
individuals  who  were  not  (Berkman  4  Breslow 
1983).  Some  researchers  suggest  that 
residential  segregation  is  linked  to  high 
mortality  levels  simply  as  a  result  of  low-SES 
characteristics.  However,  a  study  conducted  by 
Haan  and  his  colleagues  (1987)  documented  that 
social  and. physical  environmental  factors  are 
important  contributors  to  the  association 
between  low-SES  and  high  rates  of  mortality: 
residence  in  areas  characterized  by  social  and 
environmental  deprivation  (i.e.,  poverty  areas) 
was  associated  with  an  increased  risk  of  all- 
cause-mortality,  even  after  adjusting  for 
individual-level  measures  of  SES  and  other 
traditional  risk  factors. 

Our  analyses  reported  here  raise  many 
questions  and  suggest  that  the  association 
between  segregation  and  health  is  a  fruitful 
area  for  future  research.  We  found  that  the 
black-white  mortality  gap  among  adults  was 
higher  in  some  cities  than  in  others.  Future 
research  should  examine  regional  variation  in 
mortality  across  U.S.  regions.  Examining  the 
relationship  between  segregation  and  mortality 
separately  for  blacks  and  whites  and  for 
specific  major  causes  of  death,  including  heart 
disease,  stroke,  and  homicide  is  likely  to  shed 


important  new  light  on  the  nature  of  the 
association.  We  plan  to  address  these  issues 
in  future  analyses.  In  addition,  research  is 
needed  to  explain  the  lack  of  an  association 
between  segregation  and  the  black-white 
mortality  gap  for  ages  45-64.  Future  research 
is  also  needed  to  identify  the  critical  aspects 
of  segregated  neighborhoods  that  might  be 
related  to  health  and  well-being. 
Understanding  the  effects  of  racism  on  health 
is  crucial  to  assisting  public  health 
specialists  to  create  effective  programs,  in 
cities  where  residential  segregation  is  high, 
that  would  enhance  the  quality  of  life  of 
African  Americans  and  reduce  their  elevated 
rates  of  mortality. 

1  The  terms  African  Americans  and  blacks  are 
used  interchangeably. 
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The  Rite  Care  Program.   In  August 
1994,  Rhode  Island  implemented  a  manda- 
tory managed  care  program,  Rite  Care, 
under  a  research  waiver  from  the  Health 
Care  Financing  Administration.   Rite 
Care  covers  an  expanded  population  of 
low- income  residents,  including  (1)  AFDC 
recipients  and  "AFDC-like"  families,  who 
comprise  a  majority  of  persons  covered 
by  the  state's  Medicaid  program,  plus 
(2)  an  additional  cohort  of  low- income, 
uninsured  women  and  children,  called  the 
"waiver  population."   It  does  not  cover 
other  categories  of  Medicaid-eligible 
persons,  such  as  the  aged,  blind,  or 
disabled  persons;  foster  children  in  the 
state's  custody;  or  persons  who  become 
eligible  because  of  large  health  care 
expenditures . 

The  transition  of  recipients  from 
f ee-f or-service  Medicaid  has  been  phased 
in  over  a  period  of  twelve  months.   On 
enrolling,  Rite  Care  members  join  one  of 
five  private  sector  managed  care  plans 
organized  on  a  "gatekeeper"  model.   At 
full  enrollment,  Rite  Care  will  cover 
75,000  persons,  of  whom  95  percent  will 
be  women  of  child-bearing  age  and  depen- 
dent children.   The  waiver  population  is 
expected  to  total  no  more  than  10,000 
persons  at  full  enrollment. 

The  five  managed  care  plans  partici- 
pate in  Rite  Care  through  having  been 
selected  on  the  basis  of  their  response 
to  a  request  for  proposals  by  the  state. 
Of  the  five  plans,  one  is  a  staff  model 
health  maintenance  organization  (HMO) , 
three  are  independent  practice  associa- 
tion model  HMO's,  and  one  is  an  HMO  es- 
tablished by  the  state's  community 
health  centers  specifically  to  partici- 
pate in  the  program.   Except  for  the 
last-listed  plan,  all  plans  were  serving 
commercial  members  in  Rhode  Island  prior 
to  the  conception  of  Rite  Care. 

Rite  Care  members  receive  all  the 
benefits  available  under  f ee-f or-service 
Medicaid,  plus  a  package  of  enhanced 
benefits  developed  for  Rite  Care  and, 
for  eligible  mothers,  family  planning 
benefits  for  an  additional  24  months 
post-partum.   The  enhanced  benefits  in- 
clude transportation  to  and  from  health 
care  providers'  offices,  smoking  cessa- 
tion sessions,  childbirth  education 
classes,  parenting  classes,  and  nutri- 
tion counseling.   Almost  all  covered 
benefits  are  provided  and/or  paid  for  by 
the  plans,  who  receive  a  monthly  capita- 
tion payment  for  each  of  their  Rite  Care 
members.   Selected  benefits  are  carved 


out  for  payment  on  a  f ee-f or-service 
basis,  including  all  routine  dental  care 
and  mental  health  and  substance  abuse 
treatment  services  beyond  the  "in-plan" 
maximum. 

Research  and  Evaluation.   Rite  Care 
was  designed  jointly  by  the  state's 
Medicaid  and  public  health  agencies  with 
the  intention  of  achieving  improved 
health  status  as  well  as  cost-effective 
coverage  for  high  quality  medical  care. 
It  includes  a  comprehensive  research  and 
evaluation  (R&E)  component,  administered 
by  the  public  health  agency,  that  is 
integral  to  program  administration.   The 
R&E  component  will  evaluate  the  impact 
of  managed  care  on  the  health  and  health 
care  of  the  covered  population  in  four 
areas : 

Access  to  care 

Quality  of  care  and  health  status 
Reform  of  the  delivery  system 
Benefit  plan  design 

Within  each  area,  a  set  of  hypotheses 
have  been  established  that  will  be 
tested  during  the  research  phase  of  the 
program.   The  research  design  developed 
to  test  these  hypotheses  specifies  the 
measures  to  be  monitored  and  the  data 
sources  to  be  accessed  or  established. 
This  set  of  key  indicators  to  be  tracked 
includes  the  following  measures  of 
utilization  for  program  participants: 

Measures  of  women's  health 

Number  and  timing  of  family 

planning  visits 
Cervical  cancer  screening  per 

guidelines 
Breast  cancer  screening  per 

guidelines 

Pregnancy  and  birth  indicators 

Time  of  entry  to  prenatal  care 
Adequacy  of  prenatal  care 
Occurrence  of  mother's  post- 
partum visit 
Cesarean  section  rate 

Measures  of  children's  health 

Immunizations  per  schedule 
EPSDT  visits  per  schedule 
Occurrence  of  preventive  dental 
visits 

General  measures  of  health 

Frequency  of  inappropriate  emer- 
gency room  visits 

Incidence  of  preventable  hospi- 
talizations 

Continuity  of  primary  care 
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These  measures  are  available  from  a 
variety  of  existing  data  sources  and 
from  data  collection  activities  insti- 
tuted by  the  Rite  Care  program.   New 
data  collection  efforts  include  (1)  a 
periodic  interview  survey  of  low- income 
mothers,  performed  at  their  children's 
first  birthday  and  linked  to  medical 
records;  (2)  complete  person- level  data 
from  the  five  health  plans  on  all  medi- 
cal encounters  of  Rite  Care  members;  (3) 
review  of  selected  medical  records  for 
focused  studies;  and  (4)  regular  surveys 
of  samples  of  Rite  Care  members.   Exis- 
ting data  sources  to  be  accessed  include 
(1)  the  Medicaid  Management  Information 
System  established  for  the  fee-for-ser- 
vice  program,  (2)  statewide  birth  cer- 
tificates, and  (3)  statewide  hospital 
discharge  data.   For  some  measures,  more 
than  one  database  must  be  accessed  to 
provide  data  on  the  Rite  Care  population 
and  data  on  a  comparable  population, 
either  Medicaid  recipients  during  the 
period  prior  to  Rite  Care  or  commercial 
members  of  the  health  plans  participat- 
ing in  Rite  Care. 

Infant  Health  Survey.   Of  the  new 
data  collection  efforts,  the  infant 
health  survey  and  the  health  plan  en- 
counter data  have  required  the  largest 
efforts  in  development  and  implementa- 
tion.  The  infant  health  survey  is  a 
face-to-face  interview  with  a  sample  of 
women  whose  children  have  reached  one 
year  of  age.   The  sample  is  selected 
from  women  living  in  12  inner-city,  low- 
income  census  tracts  and  includes  both 
Medicaid  eligible  and  other  women.   The 
survey  content  covers  demographics,  de- 
tails of  delivery,  birth  outcomes,  regu- 
lar providers  of  care,  preventive  and 
emergency  utilization,  health  status, 
insurance  coverage,  and  barriers  to 
care.   This  information  is  also  linked 
to  data  abstracted  from  the  mother's  and 
the  child's  medical  records,  as  well  as 
to  the  data  filed  with  the  child's  birth 
certificate.   The  first  infant  health 
survey  was  a  baseline  survey  conducted 
in  1994  and  included  343  women  whose 
children  were  born  between  March  and 
July  1993;  subsequent  surveys  are 
planned  at  two-year  intervals. 

Encounter  Data  System.   At  the  core 
of  the  R&E  component  is  the  encounter- 
based  utilization  reporting  system  for 
Rite  Care.   The  encounter  data  system 
consists  of  three  distinct  databases 
reported  in  common  formats  by  all  par- 
ticipating health  plans: 

Person- level  encounter  data  for  all 
Rite  Care  members,  including  details 
of  hospital  inpatient  stays,  ambula- 
tory care  by  all  licensed  providers, 
and  expanded  data  on  prenatal  care 
and  deliveries. 


Aggregated  encounter  data  for  all 
Rite  Care  members  including  utiliza- 
tion of  ancillary  services  and  gross 
measures  of  quality  adapted  from 
HEDIS,  HCQUIS,  and  other  sources 
relevant  to  managed  care . 

Aggregated  encounter  data  for  a  sam- 
ple of  each  health  plan's  commercial 
members  including  data  comparable  to 
that  submitted  in  aggregate  for  Rite 
Care  members . 

Data  are  submitted  in  quarterly  batches, 
with  the  plans  allowed  90  days  to  pre- 
pare the  submission.   All  plans  extract 
the  encounter  data  from  their  internal 
claims  or  pseudo-claims  databases,  and 
the  delay  allows  time  for  claims  for 
services  during  a  quarter  to  be  sub- 
mitted from  providers  and  processed  by 
the  plans.   The  person- level  encounter 
file  includes  information  on  member 
characteristics,  provider  characteris- 
tics, diagnoses,  procedures,  and  charges 
and  payments.   The  aggregate  data  file 
includes  measures  of  utilization  and 
quality.   A  comprehensive  reporting  sys- 
tem developed  for  use  with  the  encounter 
database  provides  quarterly  reports  in 
the  areas  of  financial  and  actuarial 
analysis,  medical  management,  quality 
assurance,  provider  capacity,  and  re- 
porting to  HCFA,  as  well  as  research  and 
evaluation.   An  ad  hoc  capacity  has  been 
established  to  support  more  detailed  or 
unanticipated  program  needs  for  informa- 
tion. 

Because  the  encounter  data  system 
involves  the  conversion  of  data  from  the 
idiosyncratic  claims  processing  systems 
of  five  managed  care  plans  into  a  common 
format,  there  have  been  many  obstacles 
to  overcome  in  the  development  process. 
Some  key  obstacles  and  their  solutions 
are  outlined  briefly  below: 

Common  Provider  Identifiers.   Each 
plan  uses  its  own  provider  numbering 
system,  and  not  all  participating 
providers  have,  or  are  willing  to 
obtain,  an  identification  number  as  a 
provider  in  the  f ee-for-service 
Medicaid  program.   The  solution  is  to 
use  the  state's  license  number  as  a 
common  identifier,  which  has  required 
the  plans  to  obtain  these  numbers  and 
link  them  to  their  provider  records. 
Specific  problems  have  arisen  in  the 
case  of  out-of-state  providers  and 
providers  who  bill  through  a  cor- 
porate entity,  such  as  a  group  prac- 
tice . 

Global  Payment  for  Prenatal  Care  and 
Delivery.   Most  plans  pay  a  single 
comprehensive  fee  to  obstetricians 
that  covers  their  services  for  pre- 
natal care  and  delivery.   The  claim 
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that  is  submitted  has  no  information 
on  the  number  or  timing  of  prenatal 
visits,  the  birthweight  of  the  in- 
fant, the  infant's  gestational  age, 
and  other  key  information.   In  order 
to  obtain  this  information,  a  simple 
birth  worksheet  to  be  filled  out  by 
the  obstetrician  was  designed  and 
adopted  by  all  plans.   Some  plans  are 
considering  expanding  use  of  the  form 
to  their  commercial  members  in  order 
to  obtain  uniform  data  on  this  sub- 
ject for  all  their  members. 

Ambulatory  Care  Clinics  in  Teaching 
Hospitals .   Several  plans  are  using 
residents  in  hospital -based  primary 
care  teaching  clinics  to  serve  as 
members'  "gatekeeper"  primary  care 
physicians.   Claims  submitted  from 
these  clinics  do  not  identify  the 
specific  resident  providing  care.   In 
order  to  look  at  issues  of  caseload, 
practice  patterns,  and  continuity  of 
care,  the  hospital  clinics  are  pro- 
viding additional  data  files  of  resi- 
dent-specific encounters  to  the  plans 
for  submission  to  the  Rite  Care  pro- 
gram. 

In  each  of  the  above  instances,  and 
in  other  design  decisions  not  described, 
solutions  were  developed  through  colla- 
boration between  the  program  and  the 
participating  health  plans.   Consider- 
ation was  given  to  the  impact  on  the 
plans'  resources  as  well  as  the  need  to 
meet  key  objectives  of  the  program  and 
the  R&E  component.   In  these  delibera- 
tions, program  staff  assigned  high  pri- 
ority to  the  inclusion  of  data  relevant 
to  issues  such  as  provider  accountabili- 
ty and  emphasis  on  primary  care  and  pre- 
ventive services,  as  evidenced  in  the 
above  examples . 

Preliminary  Findings.   At  present, 
the  submission  of  encounter  data  from 
the  plans  is  in  the  testing  phase,  with 
the  first  round  of  full  reporting  to  be 
completed  by  September  30,  1995.   Data 
from  the  baseline  infant  health  survey 
has  been  analyzed,  however,  and  several 
findings  are  worth  presenting.   Prior  to 
the  implementation  of  Rite  Care,  low- 
income  infants  in  the  sampled  population 
were  most  likely  (49  percent)  to  receive 
regular  primary  care  at  community  health 
centers,  followed  by  hospital  outpatient 
clinics  for  pediatric  or  general  medi- 
cine (25  percent)  and  private  physi- 
cians' offices  (19  percent).  (See  Figure 
1.)   Smaller  numbers  were  seen  in  the 
staff  model  HMO's  centers  and  hospital 
emergency  rooms.   Overall,  a  large 
majority  of  these  infants  were  seen  at  a 
frequency  judged  adequate  (57  percent) 
or  better  (28  percent) ,  but  a  signifi- 
cant proportion  (15  percent)  had  fewer 
than  five  visits  during  their  first  year 


of  life.  (See  Figure  2.)   Subsequent 
iterations  of  the  infant  health  survey 
will  provide  information  on  whether  the 
Rite  Care  program  has  changed  and/or 
improved  these  utilization  patterns. 

Discussion.   The  introduction  of  man- 
aged care  for  Medicaid  recipients  pres- 
ents challenges  and  opportunities  that 
can  only  be  monitored  and  assessed  by 
the  development  and  implementation  of  a 
program-oriented  evaluation  plan.   The 
availability  of  data  is  a  critical 
determinant  of  the  scope  of  the  evalua- 
tion plan,  and  the  adequacy  of  existing 
data  sources  and  the  capacity  to  support 
the  development  of  new  sources  must  be 
realistically  evaluated  in  developing 
such  a  plan.   The  ability  and  willing- 
ness of  key  partners,  such  as  the  man- 
aged care  health  plans  in  the  Rite  Care 
program,  to  collect  and  submit  uniform 
data  must  be  determined  and,  if  neces- 
sary, enhanced  through  contract  provi- 
sions, collaborative  relationships,  and 
mutual  acceptance  of  the  importance  of 
evaluation  for  such  programs. 
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FIGURE  1 
REGULAR  SOURCE  OF  PRIMARY  CARE, 
BIRTH  TO  AGE  ONE,  1993-1994 
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FIGURE  2 

ADEQUACY  OF  PEDIATRIC  CARE 
IN  THE  FIRST  YEAR  OF  LIFE 
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MONITORING  RISK  AND  IMPACTING  UTILIZATION  OF 
PERINATAL  CARE  BY  VULNERABLE  POPULATIONS 


Dawn  K.  Aldridge,  Los  Angeles  County  Department  of  Health  Services 
Miriam  Schocken,  J.  Robert  Bragonier,  Ann  Petrus- Baker 


INTRODUCTION 

Perinatal  risk  assessment  is  designed  to 
enable  clinicians  to  anticipate  and  attempt  to 
prevent  adverse  birth  outcomes  such  as  preterm 
birth,  low  birthweight,  morbidity,  and 
mortality.   Several  standardized  prenatal  risk 
assessment  and  record  keeping  systems  have  been 
developed  to  enable  physicians  to  efficiently 
assess  risk,  maintain  a  centralized  and  concise 
patient  record,  and  design  appropriate  care 
plans.   Standardized  systems  such  as  POPRAS 
(Problem  Oriented  Perinatal  Risk  Assessment 
System) ,  the  Hollister  Prenatal  Module,  Bibbero 
Systems,  and  the  ACOG  Antepartum  Record  are 
designed  to  be  used  in  medical  offices  for 
clients  who  are  already  utilizing  care  (Gollin, 
1990) .   These  systems  tend  to  be  heavily 
medically  oriented,  technical,  and  long. 

Comprehensive  risk  assessment  and  prenatal 
care  is  frequently  inaccessible  to  women  who 
would  most  benefit  from  the  medical  care  and 
support  of  health  care  providers.   Some  of  the 
most  vulnerable  and  hardest  to  reach  women  do 
not  have  access  to  medical  and  social  services 
and  may  be  unaware  of  their  risks  in  pregnancy. 
These  women  are  not  assessed  appropriately  for 
pregnancy  risk.  They  also  may  experience 
psychosocial  and  economic  problems  that  lead  to 
stress  and  medical  problems  in  pregnancy, 
resulting  in  poor  outcomes.   Public  health 
perinatal  programs  bear  the  responsibility  of 
reaching,  assessing,  and  caring  for  these  women 
before  it  is  too  late  to  try  to  improve  their 
outcomes . 

The  primary  goal  of  the  Los  Angeles  County 
Department  of  Health  Services  Comprehensive 
Perinatal  Outreach  Program  (CPOP)  is  to  outreach 
to  the  most  vulnerable,  hardest  to  reach  women, 
to  assess  their  risks  in  order  to  encourage  and 
facilitate  appropriate  care  plans  through  case 
management.   CPOP  defines  vulnerable  and  hard  to 
reach  pregnant  women  as  women  who  lack  basic 
needs  including  medical  care,  social  support, 
shelter,  food,  and  transportation;   women  who 
may  be  experiencing  domestic  violence,  emotional 
problems,  mental  disabilities,  or  highly 
stressful  life  events;   and,  women  who  are 
substance  abusers  or  who  are  under  the  age  of 
sixteen. 

Key  components  of  CPOP ' s  outreach  and  case 
management  process  are  risk  screening  during 
outreach  and  risk  assessment  during  case 
management .    While  lengthy  and  complex 
standardized  systems  are  inappropriate  in  an 
outreach  setting,  lessons  learned  from  research 
on  standardized  indices  were  invaluable  in  the 
development  of  our  own  risk  screening  and 
assessment  tools.   We  have  modified  and  combined 
elements  from  existing  assessment  systems  to 
suit  the  needs  of  our  clients,  field  staff,  and 
research  staff.    We  have  transformed 
individually  oriented  risk  assessment  into  a 
screening  system  that  may  be  used  to  measure 


population  based  needs  and  characteristics  as 
well  as  individual  client  needs  and  risks. 
While  we  utilize  our  system  to  determine  whether 
a  client  meets  our  high  risk  criteria,  we  also 
utilize  our  data  to  describe  the  demographics  of 
our  population,  assess  the  population's 
utilization  of  and  access  to  health  care, 
improve  targeting  of  our  interventions,  and 
analyze  outcomes  and  program  effectiveness. 

PROGRAM   DEVELOPMENT 

Due  to  time  constraints  we  were  forced  to 
develop  our  policies  and  procedures  and  our  data 
collection  tools  while  staff  were  in  the  field. 
We  found  that  the  most  pragmatic  way  to 
accomplish  this  task  was  to  collapse  the 
development  of  our  outreach  and  case  management 
protocol  into  the  forms  development  process . 
During  this  time  we  created  outreach  and  case 
management  procedures,  high  risk  criteria  for 
case  management  (see  appendix  A) ,  and  all  of  the 
forms  necessary  to  implement  the  procedures  and 
collect  data.   The  procedures,  criteria,  and 
forms  were  developed  and  refined  through  many 
staff  meetings  including  heads  from  all 
disciplines:    health  education,  nursing, 
obstetrics,  social  work,  nutrition  and  program 
evaluation.   Although  this  situation  was  less 
than  ideal,  as  we  developed  our  forms  we  were 
able  to  refine  our  purpose,  define  how  our 
multidisciplinary  teams  would  collaborate,  and 
create  necessary  links  with  other  community 
agencies  and  county  health  centers . 

All  CPOP  contacts  begin  with  an  initial 
client  encounter.   Some  encounters  are  obtained 
through  referral  from  the  CPOP  toll-free 
pregnancy  hotline,  public  agencies,  public 
health  clinics  such  as  high  risk  obstetrical  or 
teen  clinics,  or  community  based  organizations. 
Many  clients  are  located  through  direct  outreach 
in  public  venues.   Most  of  our  outreach  efforts 
are  targeted  to  areas  in  Los  Angeles  County  that 
have  the  highest  rates  and/or  numbers  of  infant 
deaths  and  low  birth  weight  births . 

Upon  contact  with  a  prospective  client,  a 
community  health  worker  conducts  an  initial 
screening.   The  worker  identifies  pregnant  or 
postpartum  clients  who  appear  to  be  at  risk 
and/or  who  may  not  be  receiving  adequate  care. 
If  the  client  exhibits  any  risk  factors  she  is 
referred  to  a  public  health  nurse  for  formal 
risk  assessment  and  possible  case  management 
intake.   Referrals  for  health  and  social 
services  are  given  to  women  who  are  not 
candidates  for  or  who  decline  referrals  to  case 
management  assessment.   Health  education 
materials  are  also  distributed. 

The  public  health  nurse  interviews  each 
potential  case  management  client,  assesses  her 
risk  level,  and  decides  whether  or  not  she  will 
be  case  managed  based  on  CPOP ' s  high  risk 
criteria  and  the  nurse's  professional  judgement. 
Once  a  client  is  enrolled  in  case  management, 
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she  is  referred  to  a  prenatal  care  provider  and 
to  other  CPOP  services  as  needed,  including 
health  education,  nutrition  or  social  work 
intervention.   The  nurse  develops  a  care  plan, 
follows  up  with  the  client,  and  notes  her 
progress  in  her  file.   The  public  health  nurse 
and  the  community  health  worker  attempt  to 
contact  each  woman  at  least  twice  per  month. 

Within  a  few  weeks  after  the  case  managed 
cl:.ent  delivers  her  infant,  a  postpartum 
evaluation  is  completed.   Clients  who  need 
guidance  in  the  postpartum  period  may  be 
retained  in  case  management  for  one  year. 
Occasionally,  new  clients  are  enrolled  in  the 
postpartum  period.   Decisions  to  retain  previous 
clients  or  enroll  new  postpartum  clients  are 
based  on  case  management  criteria  for  postpartum 
clients.   All  postpartum  clients  are  provided 
with  referrals  for  pediatric  care,  family 
planning,  and  special  needs. 

FORMS  DEVELOPMENT 

CPOP's  forms  development  process  was 
inexorably  linked  to  protocol,  procedure,  and 
case  management  criteria  development .   As  we 
created  forms  we  were  in  essence  creating 
policy;   therefore,  a  great  deal  of  research  and 
tine  went  into  designing  our  forms.   Our  goals 
were  to  produce  forms  that  would  guide  staff 
through  the  outreach  and  case  management  process 
that  were  easy  to  use  and  not  burdensome  to 
clients.   Yet,  they  needed  to  be  comprehensive 
enough  to  enable  us  to  assess  risk,  describe  our 
population,  and  evaluate  the  impact  of  our 
program  on  birth  outcomes  and  patient  behavior. 

We  drew  upon  many  resources  in  developing 
our  forms,  including  literature  regarding  the 
impact  of  specific  birth  outcome  indicators, 
clinical  experience,  experience  of  other 
perinatal  programs,  and  outreach  field 
experience.    Although  we  field  tested  and 
revised  before  the  first  printing,  all  of  our 
forms  were  dramatically  revised  after  one  year 
of  field  use. 

The  Outreach  and  Referral  Form 

At  each  outreach  encounter,  a  community 
health  worker  screens  prospective  clients  using 
the  Outreach  and  Referral  Form.   This  form  was 
the  most  difficult  to  design  because  perinatal 
outreach  was  a  relatively  new  approach  for  us. 
We  lacked  sample  forms  to  guide  us .   In 
addition,  the  form  was  difficult  to  organize 
because  it  was  to  serve  multiple  purposes.   The 
outreach  and  referral  form  serves  as  an 
encounter  script  for  community  health  workers 
and  a  guide  for  making  referrals,  as  well  as  a 
preliminary  risk  screening  and  data  collection 
tool.   This  form  was  revised  several  times 
during  its  initial  development  in  order  to  make 
it  more  user  friendly.   Feedback  was  solicited 
from  field  staff  as  they  gained  field  experience 
using  the  form.   After  the  first  year  we 
condensed  the  long  seven  page  form  to  a  more 
manageable  five  pages. 

The  form  includes  seven  sections: 
enrollment,  pregnancy  status,  health  care 
referral,  financial  assistance,  needs 
assessment,  risk  assessment,  and  actions  taken. 
Instructions  to  staff  are  imbedded  in  many 
sections  of  the  form.   This  structure  enables 


the  community  health  worker  to  provide  referrals 
to  clients  throughout  the  form  within  the 
context  of  the  needs  being  discussed  while 
collecting  data  for  analysis. 

At  enrollment  demographic  information  and 
the  source  of  the  contact  is  collected.   Then, 
the  client's  pregnancy  status  is  established. 
Referrals  to  needed  prenatal  care,  postpartum 
care,  or  pregnancy  testing  are  given  as 
indicated.   Next,  the  community  health  worker 
determines  whether  the  client  needs  and  is 
eligible  for  Medicaid.   The  client  is  asked 
about  her  health  and  personal  needs .   Based  on 
her  stated  needs,  referrals  to  services  such  as 
family  planning,  health  education, 
immunizations,  and  well  baby  care,  together  with 
health  promotion  material,  are  provided. 
Finally,  the  community  health  worker  conducts  a 
preliminary  risk  screening  which  was  abstracted 
from  our  case  management  enrollment  criteria. 
Clients  who  exhibit  at  least  one  risk  factor  are 
referred  to  a  public  health  nurse  for  further 
evaluation.   After  the  interview  is  completed 
the  health  worker  notes  all  types  of  referrals, 
information,  and  assistance  he  or  she  provided 
in  the  "actions  taken"  section. 

The  outreach  and  referral  form  gives  us  a 
broad  picture  of  the  characteristics  of  the 
women  that  our  community  health  workers 
interview.   The  data  include  information  about 
all  of  the  women  that  we  contact,  including 
those  who  refuse  our  services.   Thus,  we  are 
able  to  determine  who  we  are  reaching  and  how 
well  we  are  reaching  the  high  risk  population 
within  the  target  area.   We  also  use  the  data  to 
determine  the  sources  of  contact  that  yield  the 
most  high  risk  women  and  potential  case 
management  clients.   We  use  this  information  to 
target  our  outreach  efforts  more  effectively. 
In  addition,  we  utilize  the  data  to  characterize 
the  most  common  needs  and  risks  of  our 
population.   This  information  improves  our 
ability  to  meet  the  needs  of  our  target  group. 
Finally,  supervisors  use  outreach  case  load 
information  to  identify  staff  work  patterns  and 
to  provide  statistics  for  official  reports. 

Case  Management  Intake  Form 

A  public  health  nurse  collects  more 
detailed  information  about  demographics, 
utilization  of  care,  and  risks  on  the  Case 
Management  Intake  Form  during  the  case 
management  intake  process.   While  the  outreach 
form  is  designed  for  risk  screening  and 
referral,  the  case  management  intake  form  is 
designed  for  formal  risk  assessment.    Also,  the 
intake  form  was  designed  to  be  linked  to  the 
Birth  Outcome  Form  so  that  we  could  compare 
outcomes,  such  as  birthweight  and  gestational 
age,  across  risk  levels  and  demographic 
categories. 

The  case  management  intake  form  begins 
with  demographic  questions.   We  included  factors 
that  we  considered  to  be  possible  indicators  of 
poor  outcomes  in  our  population  such  as  age, 
ethnicity,  relationship  with  the  father, 
education,  type  of  residence,  and  health 
insurance  status.   We  were  interested  in 
assessing  the  level  of  vulnerability  in  our 
population  by  using  these  indicators  and  in 
comparing  outcomes  across  these  levels. 
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Education,  type  of  residence,  and  employment  and 
insurance  status  are  seen  as  proxies  for  income 
or  financial  vulnerability.    The  status  of  the 
woman's  relationship  to  the  father  may  indicate 
her  level  of  social  vulnerability. 

We  also  asked  the  patient  about  her  entry 
into  prenatal  care  and  her  estimated  date  of 
delivery.   We  wanted  to  be  able  to  ascertain 
when  in  pregnancy  we  were  obtaining  patients, 
and  whether  they  were  already  in  prenatal  care. 
We  anticipated  that  many  of  our  clients  would 
not  yet  have  begun  prenatal  care.   In  fact,  we 
hoped  to  obtain  as  many  case  management  patients 
as  possible  who  were  not  receiving  care.   In 
order  to  help  us  understand  the  needs  of  our 
clientele  better,  we  asked  clients  who  were  not 
getting  prenatal  care  why  they  had  not  received 
care.   We  provided  fifteen  possible  barriers  as 
options  as  well  as  an  "other"  category.   We 
based  these  possible  barriers  to  prenatal  care 
upon  current  literature  and  clinical  experience. 
(Scupholme,   et  al . ,  1989;   Lia-Hoagberg,   et 


et  al . ,  1993;   Aved,  et 
al. ,  1990;   Giblin,  et 

St.  Clair,  et  al . , 
1993;   Leatherman,  et 
,  1990)   We  also 


al . ,  1990;   Cartwright, 

al .  ,  1993;   Driscoll,  et 

al.,  1990;   Hansel,  1989 

1990;   Harvey  and  Faber, 

al.  ,  1990;   Sable,  et  al 

planned  to  share  this  information  with  the 

public  health  clinic  administration,  hospital 

and  clinics  committee,  and  the  OB  Access  Task 

Force,  so  that  they  could  improve  services  to 

low  income,  high  risk  patients. 

The  risk  assessment  portion  of  the  case 
management  intake  form  has  a  two- fold  purpose: 
to  assess  the  client's  risk  for  poor  birth 
outcome  and  to  assess  the  client's  risk  for  not 
obtaining  prenatal  care.   CPOP  has  a  clearly 
de_ineated  target  group:   women  who  are  hard-to- 
reach,  who  are  socially  and  psychologically 
vulnerable,  and  who  are  at  high  risk  for  poor 
birth  outcomes.   Therefore,  it  was  critical  that 
we  determine  both  the  patient's  direct  risks  for 
poor  birth  outcome  as  well  as  her  need  for 
personalized  support  and  case  management.   The 
latter  goal  was  based  on  the  assumption  that 
clients  who  have  more  psychosocial  risks  may  be 
less  able  to  obtain  adequate  prenatal  care,  tend 
to  enter  care  later,  and  may  have  medical  and 
social  needs  that  go  unattended. 

With  these  dual  goals  in  mind  we  ruled  out 
the  use  of  any  single  standardized  risk 
assessment  form  because  standardized  forms  are 
not  designed  for  use  in  outreach  setting  and 
they  tend  not  to  assess  the  risks  of  women  with 
complex  psycho-social  needs.   Baruffi,  Strobino, 
and  Dellinger  (1984)  found  that  the  predictive 
ability  of  standardized  risk  scales  varied 
significantly  across  two  populations  of  women 
who  were  socio-economically  similar,  despite 
controls  for  population  fixed  effects.   Molfese 
(1989)  suggested  that  more  work  should  be  done 
to  determine  if  risk  scales  tailored  to  specific 
communities  yield  better  results  than 
generalized  risk  scales.   In  light  of  the  doubts 
generated  by  these  validity  studies,  we  felt 
that  it  was  necessary  for  us  to  tailor  our  risk 
scale  to  our  target  population. 

We  began  by  gathering  information  on  risk 
assessment  criteria  and  risk  scales.   We 
reviewed  prospective  standardized  scales 
includinq  the  ACOG  Antepartum  Record  and  the 


Problem  Oriented  Perinatal  Risk  Assessment 
System  (POPRAS,  1987) .   We  also  considered 
several  tools  developed  by  local  public  health 
and  community  based  perinatal  programs.   In 
addition,  we  reviewed  literature  pertinent  to 
preterm  risk,  medical  risk  and  psychosocial  risk 
(Holbrook,  Laros,  and  Creasy,  1989;   Bragonier, 
Cushner,  Hobel,  1984;  Lobel,  Dunkel-Schetter , 
and  Scrimshaw,  1992) .   Finally,  we  drew  upon  the 
clinical  experience  of  three 

obstetrician/gynecologists,  a  clinical  social 
worker,  and  several  public  health  nurses. 

The  first  iteration  of  the  intake  form  was 
long  and  complex.   We  quickly  found  that  it  was 
too  cumbersome  for  our  public  health  nurses  to 
use  efficiently.   Based  upon  field  experience 
and  further  research,  we  greatly  modified  the 
form  and  ultimately  defined  three  categories  of 
pregnancy  risk  including:   preterm  risk,  medical 
problems,  and  psychosocial  risk. 

While  developing  our  preterm  birth  high 
risk  criteria,  we  determined  that  a  partially 
weighted,  simple  method  of  assessing  clients  for 
our  high  risk  case  management  program  and  the 
use  of  a  limited  list  of  risk  factors  would  be 
most  effective.  We  based  this  decision  on  the 
work  of  Holbrook,  Laros  and  Creasy  (1989)  who 
found  that  there  was  no  predictive  advantage 
gained  in  using  a  complex  system  of  risk 
scoring.   They  concluded  that  in  lusion  in  a 
high  risk  group  based  on  the  presence  of  any  one 
major  risk  factor  or  any  two  minor  risk  factors 
was  as  sensitive  as  inclusion  based  on  an 
additive  risk  score.   They  compiled  a  list  of 
the  most  highly  correlated  risk  factors  in  their 
sample  to  create  their  "Major  and  Minor  Risk 
Factors  of  the  Modified  Scoring  System  for 
Spontaneous  Preterm  Labor."   CPOP  adopted  these 
factors  for  the  preterm  risk  criteria  and 
adopted  a  bimodal  approach  to  inclusion  of 
clients  into  the  high  risk  case  management 
program.    Any  client  who  exhibits  any  one  of 
the  major  risk  criteria,  or  two  minor  factors  as 
defined  by  Holbrook,  Laros,  and  Creasy,  is 
considered  high  risk  for  preterm  birth  and 
qualifies  for  case  management.   We  collapsed 
several  sets  of  questions  from  POPRAS  into 
single  questions  to  define  the  major  risks  for 
the  medical  problems  section  of  our  current 
intake  form.   Any  positive  answer  qualified  the 
woman  for  case  management  services . 

Each  risk  included  in  the  psychosocial 
section  of  the  intake  form  also  was  considered  a 
major  risk.   This  approach  enabled  us  to  admit 
clients  who  were  psychosocially  vulnerable,  but 
who  did  not  show  any  current  medical  risk  or 
problems.   Research  concerning  the  impact  of 
psychosocial  stressors  and  perceived  stressors 
on  birth  outcomes  supports  our  definition  of 
these  stressors  as  criteria  for  case  management 
(Bragonier,  Cushner,  Hobel,  1984;  Lobel,  Dunkel- 
Schetter,  and  Scrimshaw,  1992) .   In  addition, 
based  upon  research  regarding  barriers  to  care, 
we  knew  that  psychosocial  risk  is  negatively 
related  to  timeliness  of  care,  and,  thus,  could 
be  negatively  related  to  birth  outcomes  (Sable, 
et  al .  ,  1990).   The  data  from  this  section  also 
help  us  to  describe  and  better  understand  the 
women  we  are  serving  and  to  determine  the 
relationship  between  psychosocial  risks  and 
birth  outcomes  in  our  population. 
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After  our  first  year  of  operation  we 
revised  our  original  form,  shortening  it  from 
four  pages  to  two  pages.   We  held  discussions 
with  field  staff  who  were  using  the  form,  with 
evaluators  who  were  using  the  data  for  analysis, 
and  with  data  entry  personnel .   Tradeoffs 
between  research  goals  and  the  outreach  process 
were  made  in  shortening  the  form  and  collapsing 
several  questions  on  individual  risks  into  a 
single  question  regarding  sets  of  risks.   A  more 
complex  intake  form  could  yield  a  more  exact 
picture  of  the  individual  risks  of  each  client 
and  might  enable  us  to  estimate  impacts  of 
particular  medical  or  preterm  risks  in  our 
populations.   However,  in  the  outreach  setting, 
it  is  frequently  difficult  to  obtain  specific 
information  from  clients  on  each  of  several 
medical  conditions .   The  short  case  management 
intake  form  is  a  more  useful  tool  for 
determining  whether  or  not  a  client  should  be 
classified  as  high  risk.   We  opted  for  this 
efficient,  user  friendly  solution.   Meanwhile, 
we  also  identified  and  modified  survey  questions 
that  were  ambiguous  or  difficult  to  understand. 

ANALYSIS 
Sample 

This  sample  includes  3  77  CPOP  case  managed 
clients  who  delivered  within  the  period  of 
September  1994  to  April,  1995.   Women  who  had  a 
multiple  gestation  or  for  whom  we  did  not  have 
birthweight  information  available  were  excluded 
from  this  analysis.   Since  CPOP  is  not  primarily 
a  research  project,  we  were  not  able  to  conduct 
a  formal  case  control  study,  nor  do  we  have 
complete  demographic  records  for  each  client. 

Our  sample  is  predominantly  Hispanic 
(58%),  26%  were  African-American,  6%  were 
Caucasian,  7%  were  Asian/Pacific  Islander,  and 
3%  were  of  other  ethnicities.   A  large  majority 
(85%)  reported  that  they  were  insured  by 
Medicaid,  while  just  3%  had  private  health 
insurance  and  12%  percent  had  no  health 
insurance.   Just  over  one  quarter  (28%)  reported 
that  they  were  married,  67%  were  never  married, 
and  5%  were  divorced/separated/widowed.   Under 
half  (47%)  lived  with  their  partner.   Fourteen 
percent  of  the  clients  reported  that  they  had 
less  than  a  sixth  grade  level  of  formal 
education,  72%  said  they  had  schooling  between 
6th  and  12th  grade,  and  14%  had  studied  beyond 
high  school.   A  large  majority  (86%)  were 
unemployed,  6%  were  employed  full  time,  and  8% 
were  employed  part  time.   Nine  percent  of  our 
clients  were  under  the  age  of  sixteen,  28%  were 
aged  16  to  19,  45%  were  aged  20  to  30,  and  18% 
were  over  30  years  of  age. 

Most  clients  (82%)  had  seen  a  doctor 
during  the  pregnancy  by  the  time  of  case 
management  intake,  while  18%  had  not  seen  a 
doctor  before  intake.   All  clients  who  did  not 
have  any  prenatal  care  at  intake  were  referred 
immediately  for  care  and  entered  within  a  month 
of  intake.   We  lack  information  on  the  trimester 
of  entry  into  prenatal  care  for  70  clients  and 
the  number  of  prenatal  care  visits  for  50 
clients.   Of  those  for  whom  we  have  information, 
46%  entered  care  in  the  first  trimester,  34%  in 
the  second  trimester,  and  21%  in  the  third 
trimester.   Thirty-three  percent  had  received 
over  13  prenatal  care  visits  at  the  time  of 


delivery,  37%  had  10  to  13  visits,  23%  had  6  to 
9,  and  8%  had  fewer  than  six  visits.   Few  cases 
(4%)  entered  case  management  in  the  first 
trimester  of  pregnancy,  53%  entered  some  time  in 
the  second  trimester,  and  43%  entered  in  the 
third  trimester. 

Methods 

For  entry  into  case  management,  each 
client  must  have  exhibited  at  least  one  risk, 
however,  most  of  CPOP ' s  clients  exhibited 
several  risks.   We  want  to  know  if  the  level  of 
risk  within  our  high  risk  group  is  related  to 
birthweight  and  prenatal  care  utilization.   We 
compare  the  mean  risk  level  of  our  clients 
across  categories  of  birth  outcome  and  prenatal 
care  utilization  using  our  prenatal  risk  index. 
Birthweight  categories  include  high  (>4000gms), 
normal  (2500-4000gms) ,  low  birthweight  (1500- 
2499gms)  and  very  low  birthweight  (<1500gms) . 
We  also  compare  mean  birthweight  in  grams  across 
levels  of  prenatal  care  adequacy  to  determine 
whether  adequacy  of  care  is  linked  to  birth 
outcome  in  our  sample.   We  utilize  ANOVA  means 
test  or  Kruskal-Wallis  One- Way  ANOVA.   All  ANOVA 
results  are  tested  for  equal  variances  across 
categories  as  well  as  for  normality.   Where 
ANOVA  conditions  for  the  homogeneity  of  variance 
across  categories  are  not  met,  Kruskal-Wallis 
statistics  are  employed  rather  than  the  F- 
statistic . 

The  prenatal  risk  index  is  based  upon  our 
case  management  high  risk  criteria.   The 
prenatal  risk  index  employs  an  additive  scoring 
of  risks  where  the  weight  of  each  major  risk  is 
one,  and  each  minor  preterm  risk  as  defined  by 
Holbrook,  Laros  and  Creasy  (1989)  counts  as  half 
of  a  major  risk.   We  chose  a  simple  additive 
score  of  total  risks  rather  than  a  complex 
weighted  score  for  several  reasons:   a) 
empirical  correlations  are  not  available  for 
weighting  the  impact  of  all  of  our  risks;  b) 
when  reliable  empirical  evidence  is  not 
available,  weights  tend  to  be  arbitrary;  a) 
weighted  scales  do  not  necessarily  predict 
outcomes  better  than  unweighted  scales;  and,  d) 
an  unweighted  risk  score  is  simpler  and  more 
cost  effective  to  calculate  (Strobino  and 
Baruffi,  1984;  Molfese,  1989).   Our  prenatal 
risk  index  has  a  serious  limitation  in  that  most 
risks  are  self  reported  at  the  time  of  intake. 
Reporting  of  medical  and  preterm  risks  require 
testing  or  clients  knowledge  of  specific  medical 
information.   These  risks  are  most  likely  under- 
reported.  We  have  more  confidence  in  the  self- 
reported  psychosocial  risks.   All  CPOP  clients 
on  average  reported  a  total  of  2.75  risks,  .25 
preterm  risks,  .86  medical  risks,  and  1.65 
psychosocial  risks. 

The  Kotelchuck  index  was  chosen  as  our 
measure  of  adequacy  of  prenatal  care  because  it 
evaluates  on  the  basis  of  two  distinct  criteria: 
timing  of  the  initiation  of  prenatal  care  and 
adequacy  of  received  services  (Kotelchuck, 
1994).   A  modified  version  of  Kotelchuck's 
Adequacy  of  Prenatal  Care  Utilization  Index  was 
developed  for  this  study  with  the  help  of  Becky 
Yano,  PhD,  and  in  consultation  with  Dr. 
Kotelchuck.   We  needed  to  modify  Kotelchuck's 
tables  because  his  index  requires  the 
information  on  the  month  of  pregnancy  that 
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prenatal  care  began.   CPOP  only  collects 
information  concerning  the  trimester  that 
prenatal  care  began. 

CPOP  was  not  designed  to  be  a  research 
project;   therefore,  we  do  not  have  a  control 
group  for  this  research.   However,  we  utilize 
small  area  analysis  to  compare  our  outcomes  to 
relatively  lower  risk  populations.   In  our  small 
area  analysis  we  compare  our  birthweight 
outcomes  and  prenatal  care  utilization  to 
overall  Los  Angeles  County  statistics  and 
national  statistics,  County  statistics  are 
provided  by  the  Data  Collection  and  Analysis 
Unit  of  the  Department  of  Health  Services  of  Los 
Angeles  County.   National  statistics  are  taken 
from  the  Centers  for  Disease  Control  and 
Prevention  1992  Advance  Reports.   These  are  the 
most  recent  statistics  currently  available. 

Risk,  Prenatal  Care  Utilization,  and  Outcomes 

We  were  able  to  calculate  the  modified 
Kotelchuck  Adequacy  of  Prenatal  Care  Utilization 
Index  (APNCU)  for  307  women  in  our  CPOP  sample. 
Of  these  29%  of  CPOP  clients  had  inadequate 
prenatal  care,  2%  had  intermediate  care,  69%  had 
adequate  or  adequate  plus  care .   Adequacy  of 
care  differed  greatly  between  the  two  largest 
ethnic  groups  of  our  sample:   45%  of  African 
Americans  had  inadequate  care  compared  to  23%  of 
non-African-Americans.    This  difference  is 
statistically  significant  (p<.00l).   We  also 
found  that  inadequacy  of  care  was  most  common 
for  women  under  the  age  of  16  (40%)  and  over  35 
(41%),  while  28%  of  women  16-19,   27%  of  women 
aged  20-30,   and  18%  of  women  31-35  received 
inadequate  care . 

Using  ANOVA,  we  found  that  total  risk 
level  and  psychosocial  risk  level  were  both 
significantly  related  to  adequacy  of  prenatal 
care.   Women  who  had  inadequate  care  on  the 
APNCU  index  average  3 . 3  total  risks  while  women 
with  adequate  or  adequate  plus  care  averaged  2.6 
total  risks  (p<.001).   Women  with  inadequate 
care  also  averaged  2.0  psychosocial  risks  while 
women  with  at  least  adequate  care  averaged  1 . 5 
psychosocial  risks  (p=.002).     However, 
differences  in  the  mean  birthweight  did  not  vary 
significant  over  level  of  prenatal  care 
adequacy. 

We  also  found  that  the  mean  total  risk 
level  of  the  client  does  not  significantly 
differ  across  groupings  of  birthweight.   The 
total  risk  level  was  broken-down  into  categories 
of  risk;  preterm,  medical,  and  psychosocial. 
The  means  of  these  risk  levels  also  do  not 
differ  significantly  across  birthweight 
categories . 

Comparisons 

Since  our  sample  contains  a  solely  high 
risk  population,  it  may  be  considered  higher 
risk  for  poor  birth  outcomes  than  the  national 
and  the  county  samples.   CPOP's  clients  tend  to 
enter  care  much  later  than  most  pregnant  women 
in  the  nation  or  in  the  county.   While  21%  of 
CPOP's  clients  entered  care  in  the  third 
trimester,  only  5.2%  of  women  in  the  United 
States  and  3.4%  of  the  women  in  Los  Angeles 
County  entered  care  in  the  third  trimester  or 
not  at  all.   Over  75%  of  pregnant  women  in  the 
county  and  in  the  nation  initiated  care  in  the 


first  trimester,  while  46%  of  CPOP's  sample 
began  care  that  early.   CPOP's  overall  low 
birthweight  (LBW)  rate  of  9.3%  (<2500  grams)  is 
higher  than  in  both  the  county  (6.1%)  and  the 
nation  (7.1%).   African-Americans  experience 
much  higher  LBW  rates  than  other  ethnic  groups 
both  nationally  and  in  the  county.   African 
Americans  make  up  a  larger  percentage  (26%)  of 
CPOP's  sample  than   of  the  national  sample 
(17%) ,  or  the  county  sample  (11%) .   African- 
Americans  experience  a  LBW  rate  of  13.3% 
nationally  and  12.5%  in  the  county.   The  LBW 
rate  of  African-American  CPOP  clients  is 
comparable  at  13.5%.   The  larger  size  of  CPOP's 
African-American  sample  draws  CPOP's  low 
birthweight  rate  upward.   CPOP's  very  low 
birthweight  (VLBW,  <1500gms)  rate  compares  more 
favorably  to  national  and  county  statistics. 
CPOP's  VLBW  rate  is  .8%,  the  national  VLBW  rate 
is  1.3%  and  the  county  rate  is  1.1%. 

Discussion 

In  developing  CPOP's  case  criteria,  we 
posited  that  psychosocial  risk  would  predict  the 
level  of  the  adequacy  of  prenatal  care.   Through 
this  analysis  we  were  able  to  confirm  this 
assumption.   Our  outreach  and  referral  screening 
forms  and  our  case  management  intake  form  enable 
us  to  determine  level  of  psychosocial  risk  and 
to  enroll  vulnerable  clients,  who  are  frequently 
not  accessing  care,  even  if  they  do  not  exhibit 
medically  related  risks. 

We  are  unable  to  establish  a  significant 
link  between  the  level  of  risk  or  the  level  of 
prenatal  care  adequacy  and  the  birthweight 
outcome  within  our  sample.   This  is  largely 
because  all  our  clients  were  high  risk  and  our 
sample  lacked  the  variation  necessary  to  make 
significant  comparisons.   Therefore,  we  compared 
our  high  risk  sample  to  the  lower  risk  national 
and  county  populations  who  rarely  received  late 
care.   We  found  that  although  our  clients  had 
continuous  case  management  their  outcomes  were 
still  somewhat  worse  than  County  and  National 
levels.   However,  our  African-American  clients 
compared  favorably  to  national  and  county 
statistics.   Although  we  cannot  make  any 
definitive  judgement  on  the  cause  of  increased 
low  birth  weight  in  the  CPOP  sample,  it  appears 
that  a  major  factor  may  be  adequacy  of  prenatal 
care . 

A  major  value  of  this  study  is  that  CPOP, 
a  public  health  outreach  and  case  management 
program,   was  able  utilize  public  health  nurses 
in  the  field  to  collect  data  from  a  population 
that  is  costly  to  reach  and,  therefore,  costly 
to  study.   A  more  formal  case  controlled  sample 
should  be  performed  in  order  to  establish  more 
firmly  the  link  between  lack  of  access  to  care 
and  risk  to  outcomes . 

APPENDIX  A:   Case  Management  Protocol 
CRITERION  1:   At  Risk  For  Preterm  Birth 

MAJOR  FACTORS  (One  or  more  of  the  following) 

A.  History  of  :  preterm  labor  or  delivery, 

DES  exposure,  2nd  trimester  abortions  (2  or 
more) ,  cone  biopsy,  or  uterine  anomaly 

B.  Current  problems: 

cervix  effaced  <  1  cm.  long  (<34  wks) , 
cervix  dilated  >  1cm.  internal  os  (<34  wks) , 
abdominal  surgery  >  18  weeks  or  cerclage, 
uterine  irritability  <  34  weeks, 
polyhydramnios,  or  multiple  gestation 
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C.  Street  drug  use  this  pregnancy 

D.  Alcohol  use  this  pregnancy 
MINOR  FACTORS  (Two  or  more  factors) : 

A.  2nd  trimester  abortions  (1  or  more) 

B.  1st  trimester  abortions  (3  or  more) 

C.  Bleeding  >  12  weeks  this  pregnancy 

D.  Febrile  illness  during  this  pregnancy 

E.  Pyelonephritis  this  pregnancy 

F.  Environmental  tobacco  smoke  exposure 
CRITERION  2:   Medical  Problems  During  Pregnancy 

A.  Class  A2  diabetic  or  higher 

B.  History  or  presence  of  hypertension;  presence 
of  preeclampsia,  or  toxemia 

C.  Asthma 

D.  Sero-positive  for  HIV 

E.  Smoking 

F.  Anemia    (less   than   llg/dl) 

G.  Nutritional  Deficit: 
obese/underweight  prior  to  pregnancy, 
insufficient/excessive  weight  gain,  weight  loss 
currently,  pregnant  less  than  one  year  ago, 
experiencing  excessive  nausea/  vomiting, 
inadequate  diet,  or  pica 

CRITERION  3:   Psycho-Social  Risk  Factors 

A.  Homeless 

B.  Mental  illness  or  severe  depression 

C.  Age  less  than  16 

D.  A  major  stressful  life  event: 
death  or  severe  illness  in  family, 
immigration,  change  in  marital/partner  status, 
severe  marital  discord,  severe  ambivalence 
about  pregnancy 

E.  Severe  socioeconomic  hardship 

F.  Confirmed  or  suspected  family  violence 

G.  Lack  of  adequate  support  system 

H.    Noncompliance  with  prenatal  care  appointments 
I.     Inadequate,  unclean  or  unsafe  living  enviroment 
CRITERION  4:   Postpartum  Risk  Factors 

A.  Any  drug  or  alcohol  use  by  client  if 
breastfeeding,  OR  history  of  alcohol/drug  abuse 
by  client  or  other  household  member. 

B.  Current  tobacco  use  by  client 

C.  Preterm  infant  (born  less  than  or  equal  to  36 
weeks  gestation) 

D.  Infant  requiring  2  or  more  monthly  health 
related  visits 

E.  No  prenatal  care  prior  to  third  trimester 

F.  Client  with  emotional,  intellectual,  and/or 
physical  limitations  or  lack  of  parenting 
skills . 

G.  Current  family  violence  (may  include  child 
abuse) 

H.    Lack  of  family  /  social  support  system 
I.    Inadequate,  unclean  or  unsafe  living 

environment 
J.    Age  under  18,  with  2  or  more  children 
K.    Current  tobacco  use  by  other  household  member 
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In  1992,  an  estimated  38.5 
million  Americans  under  the  age  of 
65  years  did  not  have  health  care 
coverage  (1) .  Many  Americans  thus 
do  not  receive  health  care  for 
acute  medical  problems  or 
preventive  health  care.  Thus,  it 
is  essential  to  have  state-specific 
information  for  tracking  health 
insurance  coverage  and  the  use  of 
preventive  health  services. 

As  states  mount  efforts  to 
expand  health  care  coverage, 
understanding  the  sociodemographic 
and  economic  characteristics  of  the 
uninsured,  particularly  as  they 
pertain  to  preventive  health  care, 
is  crucial  to  developing  effective 
reforms  of  public  health  care. 
Understanding  the  degree  of 
variation  of  the  uninsured  among 
states  will  provide  data  critical 
to  develop  baseline  state-specific 
data  that  should  be  of  value  in 
monitoring  health  care  coverage,  as 
well  as  allocation  of  resources. 
In  addition,  investigation  of 
reasons  for  the  variation  of 
uninsured  persons  among  states  is 
critical  to  public  health 
professionals  as  more  states 
consider  their  own  efforts  in 
health  care  reform. 

This  report  summarizes  state- 
specific  data  from  the  1993 
Behavioral  Risk  Factor  Surveillance 
System  (BRFSS)  for  persons  aged  18- 
64  years  regarding  the  status  of 
health  insurance  for  acute  care, 
the  receipt  of  preventive  services, 
and  selected  behavioral  risk 
factors.  The  primary  objectives  of 
these  analyses  are  to:  a)  present 
state  specific  BRFSS  measures  of 
the  uninsured  and  attempt  to 
explain  the  reasons  for  variation 
in  health  insurance  coverage  among 
the  states;  b) assess  the  variation 
of  demographic  characteristics, 
risk  behaviors,  and  the  use  of 
preventive  health  services  among 
uninsured  and  insured  adults;  and 
c)  highlight  analysis  of 
supplemental  questions  added  to  the 
BRFSS  in  Minnesota  to  address 
health  care  utilization  and  source 
of  health  care  coverage. 

METHODS 

The  BRFSS   is   a  state-based, 

random-digit-dialed  telephone 

surveillance  system.  All  states 


select  a  probability  sample  of 
adults  aged  >  18  years  (2) .  To 
obtain  this  sample,  most  states  use 
a  multistage  cluster  design.  In 
1993,  49  states  and  the  District  of 
Columbia  participated  in  the  BRFSS, 
with  each  state  completing  from 
about  100  to  more  than  400 
interviews  per  month. 

We  used  responses  from  the 
BRFSS  questionnaire  that 
ascertained  if  a  person  had  health 
care  coverage.  Each  person 
interviewed  was  asked  "Do  you  have 
any  kind  of  health  care  coverage, 
including  health  insurance,  prepaid 
plans  such  as  HMOs  (Health 
Maintenance  Organizations) ,  or 
government  plans  such  as 
Medicare?".  Other  questions  from 
the  BRFSS  ascertained  demographic 
characteristics,  the  use  of 
preventive  health  services,  health 
status,  and  drinking  and  smoking 
status.  In  addition,  state  added 
questions  added  to  the  Minnesota 
BRFSS  are  discussed.  To  assess  the 
validity  of  these  data,  we  compared 
data  from  the  Current  Population 
Survey  (CPS)   (1)  and  the  BRFSS. 

To  identify  the  relationship 
between  insurance  status  and 
economic  characteristics,  we 
examined  state-specific  measures  of 
economic  deprivation  such  as 
employment  rate,  less  than  a  high 
school  education,  living  below  the 
poverty  level,  and  low  per  capita 
income  (3) .  We  also  calculated  the 
prevalence  ratio  (ratio  of 
prevalence  among  the  uninsured 
relative  to  the  insured)  for  use  of 
selected  preventive  health 
services,  self -reported  health 
status,  and  selected  health  risk 
behaviors. 

All  sample  estimates  from  the 
BRFSS  were  statistically  weighted 
to  reflect  the  civilian  population 
in  each  state.  We  used  SESUDAAN 
to  calculate  standard  errors  for 
the  state-based  estimates. 

RESULTS 

The  number  of  completed 
interviews  among  all  states  ranged 
from  1,189  to  4,361.  The  number  of 
men  ranged  from  493  to  1,771;  and 
the  number  of  women  ranged  from  696 
to  2,590  among  the  states.  The 
mean  response  rate  (calculated  as 
number  of  persons  who  completed 


339 


interviews  /  the  number  of  people 
who  were  eligible  for  interview) 
ranged  from  63%  to  96%  among  all 
states. 

The  percentage  of  persons  who 
reported  having  no  health  care 
coverage  varied  widely  across  the 
states,  from  less  than  10%  in  two 
states:  Hawaii  (7%)  and  Minnesota 
(9%)  to  more  than  20%  in  Florida 
(21%) , California  (21%),  Nevada 
(22%) , Oklahoma  (22%),  Texas  (23%), 
New  Mexico  (24%) ,  and  Louisiana 
(26%) (Table  1).  Using  Pearson's 
correlation  coefficient,  we  found  a 
significant  correlation  between  the 
state-specific  data  of  the  BRFSS 
and  the  CPS  (r=0.71,  p<=0.05) 
(Figure  1)  .  In  addition,  Figure  1 
shows  the  plot  of  the  paired 
estimates  and  a  regression  line 
between  the  estimates. 

Limited  by  the  availability 
of  data,  we  defined  vulnerable 
subgroups  as  those  groups  of 
persons  at  increased  risk  of  not 
having  health  care  coverage.  These 
person  may  include  those  of 
minority/ ethnic  groups,  those  with 
low  family  income  (less  than 
$20,000  per  year),  those  with  less 
than  a  high  school  education, 
unemployed  persons,  and  younger 
persons  (under  3  0  years  old) .  We 
identified  18  states  with  at  least 
10%  Black  residents  and  7  states 
with  at  least  10%  Hispanic 
residents.  Among  those  states,  the 
percentage  of  uninsured  Black 
persons  ranged  from  11% (±9)  in  New 
Jersey  to  35% (±6)  in  Louisiana;  the 
percent  of  uninsured  Hispanic 
persons  ranged  from  25% (±5)  in 
Florida  to  4  0% (±5)  in  Texas. 
Blacks  were  as  much  as  two  times 
more  likely  and  Hispanics  were 
three  times  more  likely  than  Whites 
to  be  uninsured  (Table  2) . 

Younger  persons  were  more 
likely  than  persons  aged  30  years 
and  older  to  report  being 
uninsured.  The  percentage  of 
uninsured  among  persons  under  age 
30  ranged  from  12%  (+4)  in  the 
District  of  Columbia  to  34% (+7)  in 
New  Mexico.  Persons  with  less  than 
a  high  school  education  were  up  to 
three  times  more  likely  than 
persons  with  at  least  a  high  school 
education  to  be  uninsured  and 
ranged  from  14% (+8) in  Hawaii  to 
56% (±7) in  Texas.  Persons  with  a 
family  income  less  than  $20,000  per 
year  were  at  up  to  five  times  more 
likely  than  those  with  more  than 
$2  0,000  per  year  income  to  report 
being  uninsured  and  ranged  from 
12% (±4)  in  Hawaii  to  48% (±6)  in  New 
Mexico.  Unemployed  persons  were  up 
to   six   times   more   likely   than 


employed  persons  to  report  being 
uninsured.  The  prevalence  of 
uninsured  among  the  unemployed 
ranged  from  21% (+10)  in 
Massachusetts  to  70% (±13)  in 
Arkansas. 

We  found  a  significant 
positive  association  between  the 
prevalence  of  being  uninsured  and 
the  economic  variables  of 
unemployment  (r=.37,  p<.01),  being 
at  or  under  the  poverty  level 
(r=.57;  p<.01),  and  having  less 
than  a  high  school  education 
(r=.30;  p=.03).  In  addition,  we 
found  a  significant  negative 
association  between  the  prevalence 
of  being  uninsured  and  per  capita 
income  (r=-0.50;  p<.01). 

Among  the  states,  uninsured 
persons  aged  18-64  years  were  40%- 
7  0%  less  likely  than  insured 
persons  to  have  had  their  blood 
cholesterol  checked  in  the  past  2 
years.  Prevalence  rates  for  the 
uninsured  ranged  from  24% (+9)  in 
Ohio  to  40% (±9)  in  Connecticut. 
Uninsured  persons  aged  40-64  years 
were  30%-90%  less  likely  than 
insured  persons  to  report  having 
had  a  digital  rectal  examination  in 
the  past  2  years  and  ranged  from 
10% (+9)  in  Arizona  to  47% (±17)  in 
Wisconsin.  Uninsured  women  aged 
40-64  years  were  30%-90%  less 
likely  than  insured  women  to  report 
have  had  a  mammogram  and  a  clinical 
breast  examination  in  the  past  2 
years,  ranging  from  14% (±12)  in 
Utah  to  66% (±18)  in  Connecticut. 
There  were  small  variations  in  the 
use  of  Pap  smear  among  insured  and 
uninsured  women  (Table  3) . 

Significant  differences 
existed  between  the  self-reported 
percent  of  fair  or  poor  health 
among  the  insured  and  uninsured. 
Among  the  uninsured,  persons  were 
up  to  three  times  more  likely  than 
insured  persons  to  report  being  in 
fair  or  poor  health,  ranging  from 
8% (±5)  in  Alaska  to  23% (±5) in  West 
Virginia  (Table  1)  .  The  uninsured 
were  at  most  8  times  more  likely 
than  the  insured  to  report  that 
they  couldn't  afford  to  see  the 
doctor  when  needed  to  due  to  cost, 
ranging  from  28% (±6)  in  Vermont  to 
53%  (±10)  in  New  Jersey. 

Reported  current  smoking  and 
binge  drinking  were  at  most  2  times 
more  likely  among  the  uninsured 
than  among  the  insured.  Uninsured 
persons  reported  current  smoking 
ranging  from  27%  (±5)  in  California 
to  48%  (±10)  in  Massachusetts. 
Binge  drinking  among  the  uninsured 
ranged  from  7% (±3)  in  Tennessee  to 
38% (±9)  in  Rhode  Island. 
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Minnesota  State-Specific  Data 

The  Minnesota  Department  of 
Health  asked  all  BRFSS  respondents 
12  supplemental  questions  about 
health  care  utilization  and 
insurance  coverage.  Employed 
persons  were  asked  a  question  to 
determine  their  occupational 
status.  Among  insured 
Minnesotans,  75% (±2)  reported  that 
their  primary  source  of  coverage 
for  health  insurance  was  from  their 
employer.  Employment  status  seems 
to  be  a  determinant  as  to  whether  a 
person  is  covered  by  health 
insurance  or  not.  While  9%  of  all 
adults  age  18-64  are  uninsured,  the 
percentages  for  those  employed  for 
wages,  self-employed,  and 
unemployed  were  7%,  21%,  and  28%, 
respectively.  The  highest  percent 
of  the  uninsured  by  occupational 
group  were  among  those  reporting 
their  occupations  as  service 
occupations  (20%) .  Examples  of 
service  occupations  include  food 
service,  child  care,  etc.  The 
second  highest  were  those 
classified  in  the  craft  occupation 
group  (17%),  e.g.,  mechanics, 
repairers,  and  construction  trade 
workers.  Only  4%  of  managers  and 
professionals  reported  being 
uninsured.  The  percentages  of 
insured  employed  persons  in  other 
occupational  groups  were  technical, 
sales  administrative  support  (6%)  ; 
farming  (15%);  and  laborers  (10%). 

Among  Minnesotans,  44%  of  the 
uninsured,  but  only  21%  of  the 
insured  reported  no  visits  to  a 
physician  in  the  past  year. 
Similarly,  46%  of  the  uninsured, 
but  only  21%  of  the  insured 
reported  no  visits  to  a  dentist  in 
the  past  year.  However,  the 
magnitude  of  difference  in  the  rate 
of  hospitalization  among  the 
insured  and  the  uninsured  was  much 
smaller,  9%  for  the  insured  versus 
8%  for  the  uninsured  adults. 

DISCUSSION 

The  percentage  of  people  who 
are  uninsured  varies  widely  among 
the  states,  from  7%  to  2  6%.  Among 
the  states,  the  uninsured  are  more 
likely  to  be  in  fair  or  poor 
health,  younger,  less  educated,  of 
minority/ ethnic  groups,  unemployed, 
and  live  in  families  with  lower 
incomes.  Moreover,  these  persons 
are  less  likely  to  engage  in 
preventive  health  practices  that 
can  be  effectively  encouraged  in 
the  primary  health  care  setting. 
Because  lack  of  insurance  is 
associated  with  limited  access  to 
important  preventive  health 
services,   successful  health  care 


reform  at  the  state  level  may 
ultimately  lead  to  improvements  of 
access  to  preventive  health 
services. 

Contributing  factors  to  the 
variation  in  health  insurance 
coverage  across  regions  of  the 
country  are  between-state 
variability  in  unemployment  rates, 
per  capita  income,  percent  of  those 
with  low  educational  attainment, 
and  percent  of  those  living  below 
the  poverty  level.  This 
variability  has  been  demonstrated 
in  our  study  by  the  strong 
correlations  between  these  factors 
and  state-specific  estimates  data 
on  characteristics  of  uninsured 
persons.  Our  findings  support 
earlier  studies  that  have  found 
lack  of  health  care  coverage  to  be 
associated  with  the  use  of  fewer 
medical  services  and  shows  some  of 
the  variability  across  the  US  (4- 
6). 

Information  derived  from 
state-specific  questions  added  to 
Minnesota's  BRFSS  can  be  valuable 
in  helping  the  state  improve  health 
insurance  coverage  for  its 
uninsured  population.  For  example, 
the  state-specific  data  from 
Minnesota  show  that  although  most 
persons  are  insured  through 
employer  plans,  certain  categories 
of  employed  and  self-employed 
individuals,  e.g.,  service  and 
crafts  occupational  group  members 
find  it  difficult  to  obtain  health 
insurance  coverage. 

The  findings  of  this  report 
are  subject  to  at  least  three 
limitations.  First,  because  the 
BRFSS  includes  only  households  with 
a  telephone,  the  findings  may 
underestimate  the  prevalence  of 
being  uninsured  among  persons  not 
residing  in  households  with 
telephones  (e.g.,  persons  living 
below  poverty  level,  less  educated 
persons,  and  the  unemployed) . 
Furthermore,  nonrespondents  to  the 
BRFSS  tend  to  be  younger  and  less 
educated  persons,  who  would  have  an 
increased  likelihood  of  being 
uninsured.  Third,  estimates  are 
based  on  self-reported  data  with 
all  the  inherent  limitations  in 
surveys  of  this  type  (e.g.,  the 
accurate  recall  of  the  use  of 
health  services  and  validation  of 
responses) .  Even  with  these 
limitations,  the  BRFSS  remains  an 
important  source  of  health-related 
data  that  allows  states  to  evaluate 
health  care  coverage  and  access  to 
health  care. 

As  more  states  implement 
health  care  reform  plans,  it  is 
essential  to  continue  surveillance 
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of  health  insurance  coverage  and  to 
examine  trends  of  health  insurance 
coverage  and  use  of  preventive 
health  services.  This  study  shows 
the  usefulness  of  the  BRFSS  in  the 
examination  of  health  care  coverage 
and  related  health  factors  among 
U.S.  adults,  particularly  at  the 
state  level.  State-based  data 
collected  by  the  BRFSS  have  already 
been  useful  in  developing  state 
health  plans  and  may  guide  policy 
decisions  at  the  state  and  regional 
levels.  In  addition,  BRFSS  data 
may  be  instrumental  in  helping 
health  professionals  target 
specific  vulnerable  subpopulations 
for  specific  intervention  and 
educational  campaigns. 
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Table  1 


Percent  of  persons  ages  18-64  years  who  were  uninsured  at  the  time  of  interview 
and  who  reported  that  their  health  status  was  fair  or  poor  among  the  insured 
and  uninsured,  by  state BRFSS,  1993 


State 


Sample 
Size 


Uninsured 
%  (+95%CI) 


Fair  or 
Insured 

%  (+95%CI) 


Poor  Health 

Uninsured 
%  (+95%  CI 


Alabama  1694  13.1 

Alaska  1383  17.0 

Arizona  1277  19.8 

Arkansas  1327  19.9 

California  3071  21.3 

Colorado  1526  18.3 

Connecticut  1425  11.5 

Delaware  1707  15.0 

Dist.  of  Cola  1249  10.1 

Florida  2276  21.3 

Georgia  1815  14.9 

Hawaii  1830  6.9 

Idaho  1444  17.9 

Illinois  1720  11.6 

Indiana  1633  14.1 

Iowa  1359  11.1 

Kansas  1170  11.8 

Kentucky  1822  19.9 

Louisiana  1312  25.6 

Maine  971  15.8 

Maryland  3560  11.7 

Massachusetts  1282  10.1 

Michigan  1999  11.4 

Minnesota  2747  9.0 

Mississippi  1268  19.4 

Missouri  1167  14.4 

Montana  939  19.2 

Nebraska  13  65  11.4 

Nevada  1507  21.7 

New  Hampshire  1234  12.6 

New  Jersey  1227  11.5 

New  Mexico  1059  23.8 

New  York  1922  16.5 

North  Carolina  1864  14.3 

North  Dakota  1378  12.8 

Ohio  1065  11.7 

Oklahoma  1148  21.8 

Oregon  2  3  62  19.2 

Pennsylvania  1868  12.6 

Rhode  Island  1438  12.0 

South  Carolina  1679  18.7 

South  Dakota  1383  12.8 

Tennessee  2447  15.1 

Texas  2078  23.4 

Utah  1507  15.6 

Vermont  1550  16.4 

Virginia  1480  13.7 

Washington  2182  16.4 

West  Virginia  1819  19.5 

Wisconsin  1259  12.4 
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Table  2 

Range  in  Variations  Among  States  in  the  Percentage  of  Uninsured 
Persons  Among  Vulnerable  Subgroups BRFSS,  1993 


%  Uninsured 
Low       Hicrh 

Rat 

io* 

Subgroup 

Low 

High 

Racial/Ethnic 

Group 

Blackf 

10.6 

35.3 

1.3 

2.4 

HispanicS 

24.5 

39.8 

1.8 

3.3 

Age  (in  years) 

18-29 

11.8 

33.6 

1.3 

2.9 

Educational  Level 

<  12  years 

14.0 

55.9 

1.3 

3.3 

Family  Income  per  Year 

<=$20,000  11.9 

Employment  Status 

Unemployed  20.9 


47.6 


69.7 


1.8 


1.3 


5.0 


5.8 


*The  percent  uninsured  in  vulnerable  subgroup  relative  to  the  comparative 
group . 

5§  Comparisons  are  made  among  18  States  with  >=10%  Black  residents  and  among  7 
States  with  >=10%  Hispanic  residents. 

Table  3 

Variation  in  the  Use  of  Preventive  Health  Services  and  Related  Health  Factors 
Among  Insured  and  Uninsured  Persons BRFSS,  1993 


%  Insured 
Low   High 


Blood  Cholesterol  Check  in  Past  2  Years 

Total  (18-64  yrs)    52.1   67.0 
Men  49.3   66.5 

Women  52.3   67.9 


Digital  Rectal  Exam  in  Past  2  Years 

Total  (40-64  yrs)    28.7   64.0      10.0 

Mammogram  and  CBE  in  Past  2  Years 

Total  (40-64  yrs)    54.6   75.6      14.0 

Pap  Smear  in  Past  3  Years 

Total  (18-64  yrs)    81.1   91.9      63.6 


%  Unii 

isured 
Hiqh 

Rat 

io 

Low 

Low 

High 

ears 
24.3 

40.3 

0.4 

0.7 

18.7 

40.7 

0.4 

0.8 

28.7 

52.2 

0.5 

0.8 

46.9 


65.9 


84.3 


0.3 


0.3 


0.8 


0.9 


0.9 


1.0 


Fair  or  Poor  Health 

Total  (18-64  yrs) 

4.9 

17.0 

7.5 

23.3 

0.8 

3.2 

Men 

4.4 

17.9 

4.5 

23.3 

0.5 

3.6 

Women 

4.4 

18.5 

3.3 

26.7 

0.4 

3.8 

Couldn't  Afford  to 

See  a 

Doctor 

Because 

of  Cost 

Total  (18-64  yrs) 

5.2 

14.1 

28.4 

52.5 

3.1 

7.5 

Men       ";  -ft' 

3.4 

11.1 

19.4 

46.4 

3.1 

8.5 

Women 

6.4 

17.6 

27.3 

61.0 

2.4 

8.3 

Current  Smoking 

Total  (18-64  yrs) 

12.9 

28.7 

26.7 

47.8 

1.4 

2.4 

Men 

13.5 

31.5 

22.9 

48.5 

1.2 

2.5 

Women 

12.4 

28.7 

20.6 

50.2 

0.9 

2.6 

Binge  Drinking 

Total  (18-64  yrs) 

4.7 

26.4 

6.5 

37.5 

1.1 

2.2 

Men 

7.4 

36.5 

7.3 

48.4 

0.6 

2.0 

Women 

2.1 

16.5 

2.7 

26.9 

0.7 

2.6 
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BASELINE  RESULTS  FOR  THE  ASSESSMENT  OF  THE  IMPACT  OF  CLINICAL  GUIDELINES  FOR  PEDIATRIC  ASTHMA  ON  MEDICAL 

CARE  UTILIZATION 


Jill  M.  Killian,  Mayo  Clinic,  Mayo  Foundation 
James  M.  Naessens,  Lynn  K.  Kelley,  Gregory  L.  Angstman 


Asthma  is  a  condition  characterized  by 
intermittent  partial  airway  obstruction  that 
can  be  triggered  by  a  variety  of  physical, 
environmental,  or  emotional  factors.   It 
affects  approximately  5-15%  of  children  under 
the  age  of  15'.   In  children  ages  4-12,  asthma 
has  been  cited  as  the  most  common  reason  for 
missed  school  days2.   In  a  study  done  in 
Rochester,  Minnesota,  home  to  the  Mayo  Clinic, 
the  incidence  of  asthma  in  children  ages  1  - 
14  doubled  between  1964  and  1983.   The  onset 
of  asthma  is  usually  during  childhood.   The 
median  age  of  onset  of  the  disease  is  3  years 
in  males  and  8  years  in  females3.   Besides 
being  subject  to  the  serious  side  effects  of 
their  disease,  many  pediatric  asthma  patients 
and  their  parents  have  doubts  about  managing 
asthma. 

A  clinical  practice  guideline  for  the 
treatment  of  pediatric  asthma  patients  was 
developed  in  1994  in  conjunction  with  the 
Institute  for  Clinical  Systems  Integration 
(ICSI),  of  which  the  Mayo  Clinic  is  a 
participating  member,  and  disseminated  to 
various  care  sites  within  the  Mayo  Clinic  for 
implementation.   The  guideline  closely  follows 
the  recommendations  of  the  National  Asthma 
Education  Program  Expert  Panel  Report  on 
Diagnosis  and  Treatment  of  Asthma''.   The  goals 
of  this  guideline  are  to  identify  and 
correctly  diagnose  asthma  in  the  pediatric  age 
group  without  significant  delay  and  to  provide 
optimal  management  of  asthma  through  patient 
and  parent  education,  avoidance  of  asthma 
triggers,  comprehensive  pharmacologic 
treatment,  and  the  establishment  of  a  personal 
patient  action  plan  for  exacerbations5.    A 
side  benefit  of  better  management  should  be 
reduced  medical  care  utilization  including 
reduced  emergency  room  visits  and  hospital 
admissions.   In  implementing  this  guideline, 
we  performed  baseline  measurements  in  order  to 
identify  areas  to  focus  potential  improvement 
efforts . 

The  Mayo  Clinic's  Rochester  campus  has 
three  primary  care  sites  where  a  pediatric 
asthma  patient  may  be  routinely  followed-- 
Community  Pediatrics,  Family  Medicine,  and  the 
Division  of  Pediatric  Allergy.   In  addition, 
an  acutely  ill  patient  may  also  be  seen  in  our 
Urgent  Care  Center,  St.  Marys  Hospital 
Emergency  Trauma  Unit  or  be  admitted  to  St. 
Marys  Hospital.   In  addition  to  being  staffed 
by  specialists,  the  pediatric  allergy  service 
employs  a  full-time  nurse  educator.   The  Mayo 
Clinic  also  has  a  patient  education  center, 
where  the  child  and  his  parents  may  go  to 
attend  a  class  designed  to  educate  the  asthma 
patient.   Any  site  may  refer  an  asthma  patient 
to  the  patient  education  center.   This  is  a 
class  intended  for  a  patient  with  a  new 
diagnosis  of  asthma. 


Methods 

All  patients  under  16  with  a  diagnosis 
of  asthma  with  a  visit  at  any  of  the 
previously  mentioned  sites  between  September 
and  December  1993  were  identified.   By 
reviewing  the  medical  charts  for  these  visits, 
we  were  able  to  identify  all  asthma-related 
visits  for  each  of  the  573  patients  during 
this  time  period.   An  asthma-related  visit 
meets  one  of  these  two  criteria:  1)  specific 
documentation  of  asthma  at  the  visit,  2) 
documented  symptoms  consistent  with  the  Asthma 
Severity  Index,  or  3)  improvement  of  the  same 
symptoms  with  P-agonist  treatment.   The  Asthma 
Severity  Index  uses  patient  estimates  of 
exacerbation  frequency,  symptoms  between 
exacerbations,  exercise  impairment,  and 
treatment  history;  combined  with  measurements 
of  peak  flow  rate,  spirometry,  or  respiratory 
rate;  to  classify  a  patient's  asthma  severity 
into  one  of  three  categories--mild,  moderate, 
or  severe.   Information  for  each  patient  was 
extracted  from  the  medical  record  by  a 
registered  nurse  on  the  severity  of  asthma, 
documentation  of  patient  education  and 
personal  action  plan,  and  whether  medications 
were  prescribed  over  the  four  months .   The 
personal  action  plan  helps  the  patient's 
parent  determine  the  child's  severity  during 
an  asthma  flare-up  using  a  combination  of  peak 
flow  or  respiratory  rate  and  symptoms.   It 
gives  the  parent  information  regarding 
medication  use  based  on  severity  and  when  to 
seek  medical  attention.   We  also  extracted 
information  relating  to  each  visit  during  the 
time  period.   The  information  included  the 
visit  site,  whether  the  visit  was  documented 
as  an  asthma  visit  by  the  physician,  and 
whether  there  was  documentation  of  office- 
based  education,  spirometry,  pulmonary 
function  testing,  and  patient  height. 

Statistical  assessments  between  groups 
of  patients  were  based  on  Chi-square  tests  for 
nominal  variables  and  Wilcoxon  rank  sum  tests 
for  ordinal  and  continuous  variables,  such  as 
age  and  number  of  visits. 

Results 

Overall,  the  573  patients  made  a  total 
of  1,119  asthma-related  visits  during  the 
four-month  study  period.   Five  patients  had  10 
or  more  visits  with  a  maximum  of  15  visits. 
In  addition,  there  were  116  documented  phone 
calls  relating  to  an  asthma  visit.   For  the 
most  part,  these  were  calls  to  a  primary  care 
physician  for  a  prescription  refill.   Seventy- 
seven  patients  were  hospitalized  a  total  of  85 
episodes  representing  almost  8  percent  of  all 
visits.   An  additional  381  visits  (34  percent) 
were  acute  care  visits  to  either  the  emergency 
room  or  Urgent  Care  Center . 
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In  order  to  compare  the  utilization 
patterns  across  primary  care  sites,  and  since 
in  our  practice  patients  are  not  assigned  to  a 
primary  care  practitioner,  we  categorized  the 
patients  by  the  first  primary  care  site  they 
visited  during  the  timeframe.   With  this 
classification,  42  percent  of  the  patients 
were  managed  by  the  pediatric  allergy 
specialists,  24  percent  were  followed  by  the 
generalists  in  Community  Pediatrics,  8  percent 
were  followed  by  Family  Medicine,  and  25 
percent  received  all  of  their  asthma  care 
during  the  four  months  in  acute  care  settings 
(Table  1).   However,  these  patients  may  have 
received  primary  care  outside  of  the  Mayo 
system,  which  we  did  not  collect.   A  small 
group  of  patients  was  seen  at  multiple  primary 
care  sites.   These  were  primarily  patients  who 
were  referred  to  Pediatric  Allergy  from  a 
generalist . 

Patient  gender  was  not  significantly 
different  based  on  first  primary  care  site. 
However,  age  did  vary  significantly  between 
groups  of  patients  (p  =  0.0009)  .  Only  one  of 
the  16  patients  less  than  age  one  was  seen  in 
Pediatric  Allergy,  while  infants  aged  less 
than  one  year  made  up  4  -  5  percent  of  each  of 
the  other  three  groups.   Community  Pediatrics 
had  a  significantly  higher  proportion  of  1  -  4 
year  olds  (42  percent  vs.  22  -  29  percent  for 
the  other  groups) . 


Unfortunately,  the  severity  of  asthma 
was  not  documented  sufficiently  to  compare  the 
type  of  asthma  patient  being  seen  at  each 
site.   In  an  attempt  to  assess  the  severity, 
we  looked  at  medication  use  and 
hospitalizations  for  each  of  the  patient 
groups.   The  percent  of  patients  on  medication 
ranged  from  96  to  98  percent  across  the  four 
sites  and  the  percent  of  patients  hospitalized 
in  the  four  months  ranged  from  13.3  to  13.5 
percent . 

The  rate  of  visits  per  patient  during 
the  timeframe  was  significantly  different 
across  the  four  groups.   Both  Family  Medicine 
and  Community  Pediatrics  patients  averaged  2.5 
total  visits,  Pediatric  Allergy  patients 
averaged  1.9  visits,  while  those  without 
primary  care  contacts  averaged  1.3  total 
visits.   Meanwhile,  all  the  visits  by  those 
without  primary  care  were  acute  visits,  while 
Community  Pediatrics  and  Pediatric  Allergy 
patients  averaged  0.6  acute  visits  (Graph  1). 
Furthermore,  over  two-thirds  of  the  acute 
visits  made  by  patients  attributed  to 
Pediatric  Allergy  were  made  prior  to  the 
Pediatric  Allergy  visit.   Over  60  percent  of 
the  acute  visits  by  Family  Medicine  patients 
and  50  percent  by  Community  Pediatric  patients 
were  made  after  the  primary  care  visit.  When 
directly  comparing  Pediatric  Allergy  patients 
to  those  assigned  to  either  Family  Medicine 


Table  1:   Patients  Categorized  by  First  Primary  Care  Site 


Site 

N 

% 

%  Male 

Age  (years) 
Mean  +  s . d . 

Pediatric  Allergy 

244 

43 

59.0 

7 . 6±4 . 0 

Community  Pediatrics 

141 

24 

57.4 

5 . 8+4 . 2 

Family  Medicine 

45 

8 

64.4 

6.9+4.0 

None 

143 

25 

66.4 

7 . 1+4 . 6 

Graph  1:   Utilization  by  First  Primary  Care  Site 
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or  Community  Pediatrics,  the  Pediatric  Allergy 
patients  had  significantly  fewer  total  visits 
(1.91  vs.  2.55,  p<0.001),  but  not 
significantly  fewer  acute  visits  (0.63  vs. 
0.72,  p=0.520)  . 

Utilization  of  acute  care  services  were 
also  compared.   Patients  without  primary  care 
and  those  in  Family  Medicine  had  similar 
numbers  of  visits  to  Urgent  Care  and  the 
emergency  room,  while  those  followed  by 
Pediatric  Allergy  and  Community  Pediatrics  had 
more  emergency  room  visits  than  urgent  care 
visits  (Graph  2).   Mean  Urgent  Care  visits  and 
emergency  room  visits  per  patient  were 
significantly  lower  in  the  two  Pediatric 

groups . 

To  investigate  whether  the  lower 
utilization  among  Pediatric  Allergy  patients 
is  due  to  better  management  of  the  patient's 
asthma  care,  we  examined  the  documentation  of 
care  provided  to  the  patients.   As  noted 
earlier,  the  goals  of  the  pediatric  asthma 
guideline  include  increased  patient  and  parent 


education,  the  establishment  of  a  personal 
action  plan,  and  comprehensive  pharmacologic 
treatment.   Each  of  these  goals  is  driven  by 
the  patient's  asthma  severity.   Although  there 
was  much  room  for  improvement,  16.4  percent  of 
the  patients  followed  by  Pediatric  Allergy  had 
their  severity  documented.   This  is  a 
significantly  higher  level  of  documentation  of 
severity  compared  with  6.7  percent  in  Family 
Medicine,  4.3  percent  in  Community  Pediatrics, 
and  0.7  percent  in  those  without  primary  care. 
In  addition,  the  Pediatric  Allergy  patients 
were  the  only  ones  with  any  referrals  to  the 
patient  education  center  and  with  personal 
care  plans  documented.   When  visit-level 
documentation  was  reviewed,  visits  with 
Pediatric  Allergy  had  significantly  higher 
rates  of  office-based  education  (p  <  0.000H; 
documentation  of  the  visit  as  an  asthma  visit 
by  the  physician  in  the  patient's  medical 
record  (p  <  0.0001),  and  respiratory  testing 
(p  <  0.0001).   (Table  2)   As  these  are 
important  components  of  a  management  plan, 


Graph  2:   Utilization  of  Acute  Care  Services 
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Table  2:   Visit-Level  Documentation  Across  Sites 


%  of  visits  with 

office-based 

education 

%  of  visits 

documented  as 

asthma 

%  of  visits  with 

respiratory  testing 

documented* 

Family  Medicine 

0 

72.6 

4.8 

Community  Pediatrics 

1.2 

68.1 

9.0 

Pediatric  Allergy 

10.8 

96.0 

29.6 

Urgent  Care  Center 

0 

50.6 

5.4 

Emergency  Room 

0 

71.  0 

20.2 

Hospital 

0 

76.2 

16.5 

''respiratory  testing  includes  either  pulmonary  function  test  or  spirometry 
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more  frequent  documentation  probably  coincides 
with  a  higher  level  of  attention  and  better 
management . 

Discussion 

Our  study  has  several  limitations.   In 
the  course  of  treatment  of  a  pediatric  asthma 
patient,  a  four-month  window  of  visits  is  a 
relatively  short  timeframe  for  analysis.   The 
seasonal  aspects  of  the  disease  were  certainly 
not  captured.   In  particular,  some  of  the 
patients  without  primary  core  visits  may  have 
had  a  regular  source  of  care  outside  of  our 
study  period  or  outside  of  the  Mayo  system. 

Our  comparison  of  utilization  by 
provider  group  was  based  on  the  assignment  of 
all  utilization  to  the  first  primary  care  site 
in  which  the  patient  was  seen.   It  is  not 
based  on  the  patient's  identified  regular 
source  of  care.   When  examining  utilization 
rates,  we  do  believe  that  this  may  be  a 
reasonable  classification  of  patients  in  that 
the  Pediatric  Allergy  patients  had  a  majority 
of  acute  care  visits  prior  to  their  first 
primary  care  visit.   This  suggests  that  many 
of  these  patients  may  not  have  previously  had 
a  regular  primary  care  provider,  or  they  may 
have  been  newly  diagnosed  asthma  patients. 
Once  asthma  care  is  managed  by  Pediatric 
Allergy,  it  appears  that  total  utilization  is 
reduced. 

Finally,  due  to  the  lack  of  severity 
documentation,  we  were  unable  to  adjust  for 
differences  which  we  suspect  exist  between  the 
patients  being  treated  by  the  different 
provider  groups.  Severity  is  expected  to  be 
highly  correlated  with  the  rate  of  visits, 
particularly  for  acute  exacerbations .   In 
addition,  the  guideline  calls  for  different 
actions  based  on  the  level  of  severity. 
Patients  with  mild  asthma  do  not  require  the 
level  of  education  and  management  that  is 
necessary  for  patients  with  moderate  to  severe 
asthma.   The  other  related  factor  that  would 
have  an  impact  on  utilization  that  we  did  not 
collect  is  the  duration  of  disease  since 
diagnosis.   Newly  diagnosed  asthmatics  are 
expected  to  have  more  frequent  visits  until 
the  disease  is  adequately  managed. 

These  measures  support  the 
implementation  of  the  pediatric  asthma 
clinical  practice  guideline.   Each  site 
analyzes  its  own  data  and  compares  their 
pediatric  asthma  patient  management  process  to 
that  of  the  guideline.   The  guideline  suggests 
goals  for  managing  the  disease.   Each  site's 
implementation  team  uses  these  measurements  to 
focus  their  improvement  efforts. 

In  the  early  stages  of  implementation, 
one  of  the  primary  care  sites  put  its  emphasis 
on  two  points:   1)  Patients  with  moderate  or 
severe  asthma  should  be  referred  to 
patient/parent  education  programs,  and  2) 
Patients  with  urgent  care  or  emergency  room 
visits  for  asthma  should  have  a  follow-up 
visit  with  their  regular  source  of  care  within 
eight  days  of  acute  care.   Preliminary  data 
suggests  that  these  areas  both  have  continuing 
potential  for  improvement  and  would  benefit 
the  patient. 


In  summary,  we  have  seen  that  pediatric 
asthma  patients  make  a  large  number  of 
contacts  with  the  medical  care  system  for  both 
primary  and  acute  care  services.   The  rate  of 
visits  appears  to  be  different  for  patients 
who  are  seen  by  different  providers.   The 
provider  group  that  most  closely  follows  the 
guideline,  Pediatric  Allergy,  also  has  the 
lowest  utilization  of  allergy- related  visits. 
We  have  also  seen   in  the  guideline 
implementation  process  that  groups  of 
providers  are  more  actively  engaged  in  the 
quality  improvement  process  and  are  able  to 
focus  their  efforts  when  they  have  data 
available  that  relates  directly  to  their  own 
performance . 

References 

1.  American  Lung  Association,  1975. 
Introduction  to  Lung  Diseases  (6th 
edition) ,  New  York. 

2.  Davis  DJ,  1972.  NIAID  initiatives  in 
allergy  research.  Journal  of   Allergy 
and  Clinical  Immunology,  49:323-328 

3.  Yunginger  JW,  Reed  CE,  O'Connell  EW, 
Melton  LJ,  0' Fallon  WM,  Silverstein  MD, 
1992.   A  Community-based  Study  of  the 
Epidemiology  of  Asthma,  Incidence  Rates, 
1964-1983. 

American  Review  of  Respiratory  Disease, 
146:888-894. 

4.  National  Asthma  Education  Program  Expert 
Panel  Report.   Guidelines  for  the 
Diagnosis  and  Management  of  Asthma. 
Publication  No.  91-3042.   U.S. 
Department  of  Health  and  Human  Services, 
Public  Health  Service,  National 
Institutes  of  Health,  August  1991. 

5.  Institute  for  Clinical  Systems 
Integration,  1994.   Health  Care 
Guideline:  Asthma  in  Children. 


348 


Third  Through  Fifth 
Plenary  Session 

SIational  Committee  on  Vital 

and  Health  Statistics 

A  45th 

Anniversary 

Symposium 


* 


NATIONAL  COMMITTEE  ON  VITAL  AND  HEALTH  STATISTICS' 
45TH  ANNIVERSARY  SYMPOSIUM 


Third  through  Fifth  Plenary  Sessions 


This  portion  of  the  program,  together  with  the  Sixth  through  Eighth  Plenary 
Sessions,  will  be  printed  under  separate  cover.   All  conference  participants 
will  receive  a  copy. 


351 


■'■■■■ 


■ 


Session  U 


Building  an  Improved 

Information 
Infrastructure 


I — |44M4ApN[ 


wwywywwwwiwtn 


aJRAfQP 

<  \  >f         _ 


rvE 


M 


*■:. 


as 


BUILDING  A  CORE  DATA  SET  AND  INFORMATION  INFRASTRUCTURE 
IN  A  PUBLIC  HEALTH  AGENCY 


Michael  G.  Medvesky 

New  York  State  Department  of  Health 

Patricia  A.  MacCubbin,  Lloyd  F.  Novick 


The  New  York  State  Department  of  Health 
(NYSDOH)  has  been  moving  toward  the  goal  of 
making  public  health  information  more  accessible 
and  useable  for  public  health  providers.  This 
undertaking  involves  providing  electronic  access 
to  health  information  to  counties,  local 
providers,  community-based  organizations  and 
consumers  through  the  development  and  support  of 
an  information  communication  infrastructure. 

Background 

In  the  past,  the  NYSDOH  has  seen  a  wide 
variation  in  access  to,  and  use  of,  health 
information  across  local  health  departments  and 
other  public  health  practitioners  in  New  York 
State.  Smaller  county  health  departments  without 
planning/evaluation  staff  were  at  a  disadvantage 
in  collecting  and  presenting  health  information 
needed  for  health  surveillance,  program  needs 
assessment,  and  monitoring  and  evaluation 
activities.  Even  the  larger  county  health 
departments,  with  planning/evaluation  staff 
available  for  these  tasks,  often  utilized 
information  that  was  out-of-date,  with  more 
current  data  being  available. 

County  health  departments  receive 
reimbursement  from  the  NYSDOH  for  public  health 
activities.  As  part  of  this  reimbursement 
process,  each  county  health  department  is 
required  to  prepare  a  biennial  "community  health 
assessment"  that  presents  the  demographics  of  the 
county's  population,  details  progress  on  various 
health  status  indicators,  and  identifies  service 
needs  and  gaps  in  services.  In  the  past  NYSDOH 
generated  and  circulated  hardcopies  of  the  data 
necessary  to  complete  the  assessment .  Local 
health  departments  utilized  and  interpreted  this 
information.  This  cumbersome  process  left  much 
to  be  desired.  A  Robert  Wood  Johnson  Information 
for  State  Health  Policy  grant  provided  funding 
for  the  state  to  research  and  develop 
applications  for  the  Internet.  This  provided  the 
opportunity  to  make  relevant  health  information 
available  electronically  to  the  public  health 
community . 

More  recent  efforts  in  developing  a  health 
information  infrastructure  can  be  classified  into 
two  components  : 

•  Making  health  information  more 
useable  and  accessible  to  the  public 
health  and  health  care  providers  and 
to  consumers; 


Developing    and 
info  rma  t  i  on 
infrastructure . 


supporting    an 
communications 


Health  Information  More  Useable  and  Accessible 

NYSDOH  Information  System  Inventory 

In  an  effort  to  make  health  information  more 
useable,  NYSDOH  needed  to  know  exactly  what  data 
were  being  collected  throughout  the  department. 
The  NYSDOH  conducted  an  inventory  of  existing 
information  systems.  The  resulting  information 
was  assembled  in  the  NYSDOH  Information  System 
Inventory.  Topics  covered  in  the  inventory 
include : 

•  Information  systerrvdcta  set  name 

•  System/data  set  owner 

•  Contents  of  system/data  set 

•  Demographic  informal  ion  available 


Geographic  information  available 

Data  items  available  for  possible 

record  linkage  activities 

Level  of  data 

Timeliness 

Data  collection  method 

Data  utility/data  limitations 

Restrictions  for  the  use  of  the  data 

Any  public  reports  utilizing  this 

information  system/data  set 

Contact  person  for  questions  about 

this  information/data  set 


The  inventory  will  be  available  electronically  on 
the  NYSDOH  Gopher  server. 

NYSDOH  Public  Gopher  Server  (Internet  Access) 

The  Robert  Wood  Johnson  Information  for 
State  Health  Policy  grant  supported  the 
Department's  efforts  to  make  health  information 
accessible  via  the  Internet  through  a  NYSDOH 
Gopher  server.  The  NYSDOH  public  Gopher  server 
is  menu  driven  and  part  of  the  Department's 
public  access  network.  Health  data  are  available 
in  prepared  files  on  the  Gopher.  The  NYSDOH 
Gopher  provides  public  access  to  any  NYSDOH 
information  that  can  be  requested  through  the 
Freedom  of  Information  Law.  Data  queries  are  not 
possible  through  this  server.  Files  are 
available  in  a  variety  of  formats: 


Text 

Spreadsheet 

Graphic 


In  addition  to  consumer  health  information 
(hotline  numbers,  disease  fact  sheets,  health 
service  sites)  and  official  NYSDOH  reports  and 
publications,  a  core  data  set  is  available 
electronically  on  the  NYSDOH  Gopher  server.  This 
data  set  includes  Healthy  People  2000  22.1  Core 
Indicators,  other  Healthy  People  2000  indicators, 
program-identified  process  and  outcome  measures, 
county-specific  trend  information  for  specific 
health  indicators,  county  summary  tables  of 
multiple  health  indicators,  and  small  area  (zip 
code)  information  on  selected  health  indicators. 

Health  Information  Server (s)  (Data  warehouse) 

While  publicly  accessible  health  information 
will  be  made  available  via  the  NYSDOH  Gopher 
server  on  Internet,  a  Health  Information  Server 
(HIS)  is  being  developed  to  serve  as  a  common 
electronic  repository  of  "sensitive"  health 
information.  Automated  databases  on  the  HIS  will 
be  populated  by  appropriate  "extracts"  from 
multiple  central  data  sets  constructed  with 
information  supplied  by  local  health  departments, 
other  local  providers,  or  from  state  or  federal 
data  collection  activities.  The  HIS  will  be 
available  via  the  Health  Information  Network 
(HIN) ,  thus  allowing  the  Department  the  ability 
to  limit  access  and  provide  the  necessary 
security. 


Primary    and    Preventive    Care 
Information  System  Application 


Geographic 


The  first  major  HIS  application  will  be  the 
Primary  and  Preventive  Care  Geographic 
Information  System  (PPCGIS)  .  The  PPCGIS  software 
is  being  developed  as  a  data  query/analytic  tool; 
the  HIS  "health  data  warehouse"  will  be  the 
source  of  the  health  information  for  the  PPCGIS. 
The  major  design  features  of  this  application 
include : 
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Prepackaged,  affordable  software  that 

will  operate  on  a  user's  PC 

User-friendly      interface      and 

operation,  including  on-line  help 

Ability    to    handle    different 

geographical    subdivisions,    e.g., 

county,  zip  code 

Data    selection    and    extraction 

capability,   including  selection  of 

combination  of  areas 

Ability  to  generate  tables  based  on 

variable  and  area  selection 

Ability  to  generate  graphs  based  on 

data  selection 

Ability  to  handle  data  input  on  site 

Mapping  capability 

Regular  program  and  data  updates 

Evaluation  and  revision  of  the  system 


As  mentioned,  the  Primary  and  Preventive 
Care  Geographic  Information  System  will  be 
accessible  via  the  HIN,  which  is  a  CDC 
Information  Network  for  Public  Health  Officials 
(INPHO)  Project.  Other  highlights  of  the  PPCGIS 
include : 


fax  documents  to,  the  health  care  facilities  on 
the  dial-up  mail  connects  or  Internet/Bitnet 
international  networks.  For  example,  the  Bureau 
of  Communicable  Disease  Control  sends  Problem 
Alerts  through  e-mail  and  bulk  fax. 

NYSDOH  has  also  implemented  an  external 
access  network,  StateLine,  as  a  "meeting  place" 
for  peer  connections  to  other  state  agencies  and 
for  gateway  services  to  the  entire  New  York  State 
mainframe  network,  including  the  State  Education 
network,  connecting  elementary  and  secondary 
schools  and  libraries  throughout  the  state. 
StateLine  includes  an  internal  connection  and 
Gopher  servers . 

Public  Access  Network 

The  Public  Access  Network  consists  of  the 
NYSDOH  Gopher  server,  described  previously,  and 
access  to  the  Internet  and  the  World-Wide  Web,  as 
seen  in  the  figure.  This  figure  illustrates  the 
one-way  access  feature  of  the  design. 

Health  Information  Network  (HIN) 


Client-Server  application,  where  the 

server   hardware   and   software   are 

components  of  the  HIN. 

Server  will  provide  access  to  a  very 

large   database   containing   a   wide 

range   of   preprocessed   health   and 

sociodemographic  data  organized  at 

multiple  levels  of  geography. 

Data  will  be  accessible  to  counties, 

providers,  advocates,  NYSDOH  staff, 

and   others   involved   in   health 

planning . 

Access   to   the   server   will   be 

accomplished  through  dial-in  lines 

and  direct  connection. 


Development   and   Support   of 
Communications  Infrastruture 


an   Information 


The  Public  Health  Information  System  is 
comprised  of  three  parts,  HEALTHCOM,  NYSDOH 
Gopher  and  Health  Information  Network. 

HEALTHCOM / E -Ma  i 1 

HEALTHCOM  is  an  internal  NYSDOH  wide  area 
network.  HEALTHCOM  functions  as  the  internal 
communications  system  for  the  NYSDOH  Albany-based 
staff,  Regional  Offices  and  Statewide  District 
Offices.  It  includes  file  transfer,  login, 
printing,  access  to  health  care  data,  research 
resources  (such  as  the  Internet),  office 
automation  functions,  e-mail  exchange  with  every 
health  care  facility  in  New  York  State,  and 
client/server  support  for  nearly  2,400  internal 
Department  computer  hosts .  The  HEALTHCOM  network 
supplies  appropriate  health-related  statistics 
and  other  information  products  to  the  public 
sector  via  the  Department's  Internet  Gopher. 

The  external  e-mail  services  on  HEALTHCOM 
support  4,000  dial-up  connects  to  hospitals, 
nursing  homes,  diagnostic  and  treatment  centers, 
and  clinical  laboratories  throughout  the  state. 
This  e-mail  network  provides  a  bidirectional  data 
highway  for  a  variety  of  electronic  batch-type 
applications  used  for  automated  data  collection 
and  distribution  between  the  Department  and  these 
facilities.  Data  collection  includes  health  care 
finance,  vital  records,  disease  and  heavy  metals 
registries,  health  systems  management  and 
regulations,  and  clinical  laboratory  evaluation. 
For  example,  77%  of  all  State  birth  certificates 
and  96%  of  all  State  child  blood  lead  test 
reports  are  collected  via  e-mail. 

Another  feature  of  the  e-mail  network  is 
that  it  is  designed  to  be  contiguous  between 
internal  and  external  networks,  so  that  any  user 
on  a  peer  host  computer  in  the  NYSDOH  internal 
network  can  directly  exchange  mail  with,  or  send 


The  Centers  for  Disease  Control  and 
Prevention  has  awarded  NYSDOH  a  three  year 
Information  Network  for  Public  Health  Officials 
(INPHO)  grant  to  extend  the  existing  NYSDOH 
network  infrastructure  to  county  health 
departments.  The  HIN,  a  subnetwork  of  HEALTHCOM, 
will  establish  a  standardized,  real-time,  data 
communications  network,  linking  the  county  and 
state  health  departments.  Participants  on  the 
HIN  will  have  access  to  the  NYSDOH  public  access 
network  and  to  Internet  (outbound  only), 
including  access  to  CDC.  This  will  ultimately 
include  services  such  as  Gopher,  telnet,  file 
transfer,  and  mosaic. 

The  HIN  will  include  centralized  Health 
Information  Servers  to  support  data  submission 
and  distribution  for  the  counties  on  the 
subnetwork.  Ultimately,  the  server  will  support 
a  relational  data  base  and  the  data  entry/query 
functions  needed  to  integrate  existing  internal 
health  record  databases  with  input  from 
immunization  records  and  other  key  public  health 
programs.  When  fully  operational,  the  server 
will  allow  participants  to  access  and  submit 
health  data  on  the  subnetwork  using  various 
methodologies  such  as  client/server  applications, 
Gopher,  direct  login,  file  transfer  and  e-mail. 

The  HIN  will  provide  greatly  increased 
information  exchange  with  the  county  health 
departments  and  health  providers  in  NYS,  and 
provide  Internet  access  for  county  health 
departments,  giving  them  access  to  NYSDOH  public 
information  products  on  the  NYSDOH  Gopher  and 
Internet-based  applications  at  the  CDC.  The  HIN 
will  also  support  a  centralized  immunization 
registry  linked  to  other  key  NYSDOH  health  data 
systems  on  the  Health  Information  Server  .  The 
server  will  deliver  appropriate  levels  of  data 
exchange  and  access  to  its  data  bases  for  local 
health  units,  private  providers,  and  State  and 
Federal  agencies.  Larger  counties  will  access 
the  HIN  through  dedicated  connections.  Smaller 
counties  will  have  HIN  access  through  a  1-800- 
number  dial-up  number.  By  the  end  of  the  first 
year  of  this  three-year  grant,  we  expect  to  have 
20  counties  on  the  HIN. 

Future 

The  NYSDOH  is  committed  to  continued  support 
and  expansion  of  the  NYSDOH  Gopher  and  PPCGIS. 
Local  information  networks,  such  as  those 
developed  by  diagnostic  and  treatment  centers, 
hospitals,  providers,  etc.,  will  be  utilizing  the 
Health  Information  Network.  Existing  information 
systems  will  move  to  the  HIN  for  data  transfer 
and  analysis.  The  HIN  environment  will  be  used 
for  health  education  and  training  purposes. 


356 


Acknowledgements : 

The  authors  would  like  to  acknowledge  the 
contributions  of  Dr.  Ivan  Gotham,  Frank  Kuwik, 
Colene  Byrne,  Anne  Conrad,  and  Cynthia  Gorzynski. 


HEALTH  INFORMATION  NETWORK: 

Dr.    in  Gotham,  Director 
Bureau  of  HEALTHCOM  Network 

Systems  Management 
518:473-1809 
ijgOl ©health . state .ny. us 


New  York  State  Department  of  Health 
Contact  Phone  Numbers  and  E-mail  Addresses 


OVERVIEW: 


Michael  Medvesky,  Director 
Office  of  Public  Health 
Information  Group 
518:474-2543 
mgm06@health. state .ny .us 


PRIMARY  &  PREVENTIVE  CARE  GEOGRAPHIC 
INFORMATION  SYSTEM: 

Colene  Byrne,  Coordinator 
Information  for  State  Health 

Policy  Project 
518:473-4708 
cmb04@health. state .ny. us 


NYSDOH  Gopher: 


gopher. state . health. ny . us 
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HEALTHCOM  Network 


Connections  Between  HEALTHCOM  & 
Health  Information  Network  Segments* 


Health  Information  Network  (HIN) 


BHNSM-ISHSG  /  New  York  Slate  Department  of  Health 


NYSDoH  Gopher  and  database  Server 

(gopher.health.  state,  ny.us) 
(NYSDoH  Health  Information  products 
Appropriate  for  Public  Distribution) 
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LESSONS  FOR  NATIONAL  STANDARDS  ON  HEALTH  DATA: 
STATE  PRACTICES  IN  BUILDING  DATABASES  ON  HOSPITAL  INPATIENT  STAYS 

Anne  Elixhauser,  Agency  for  Health  Care  Policy  and  Research 
Roxanne  Andrews,  Judy  Ball,  Rosanna  Coffey,  Meg  Johantgen,  Pat  Purce^l 


States  with  data  systems  are 
requesting  leadership  on  standards 
and  information  system  design  to 
improve  data  systems,  collect 
comparable  information,  and 
eventually  connect  states  to  a 
national  information  system. 
States  without  data  systems  are 
asking  for  guidance  on  developing 
them.   The  opportunity  exists  to 
help  states  adopt  standards  that 
move  the  nation  closer  to  a  uniform 
health  care  information  system.   To 
understand  the  effects  of  health 
policy,  policy-makers  and 
researchers  need  data  that  is 
comparable  across  states. 

This  project  attempts  to 
enlighten  the  development  of  data 
standards  by  exploring  their  use  in 
hospital  inpatient  data.   We 
compared  states'  data  elements  with 
standards  that  have  been 
promulgated:  UHDDS .  proposed  by 
NCVHS  and  accepted  by  US  DHHS  for 
hospital  discharge  data;  UB-92 . 
required  by  HCFA  for  Medicare 
reimbursement;  and  OMB  Directive 
15 .  a  standard  for  collecting 
information  on  race/ethnicity. 

The  sources  of  information  for 
this  study  were  1993  data  layouts 
and  other  documentation  from  12 
states/hospital  associations  with 
hospital  inpatient  databases.   This 
study  found  considerable  diversity 
in  the  variables  collected  and  in 
their  coding.   For  example,  all 
states  collected  total  charges  and 
eight  included  detailed  charges  but 
at  different  levels  of  aggregation, 
reducing  comparability  across 
states . 


The  greatest  variation  was 
found  in  the  coding  of  variables. 
Coding  for  expected  pay  source  was 
especially  diverse:  in  5  of  the  12 
states  it  was  impossible  to  isolate 
Blue  Cross/Blue  Shield  as  a  payor. 


Variations  in  Coding  of  Payor 
Source 

Pay  Source 

Number 

of 
States 

Medicare 

12 

Medicaid 

12 

Other  government 

1 

Workers'  compensation 

9 

Blue  Cross/Blue  Shield 

7 

Commercial  insurance 

11 

HMO 

6 

Self-pay 

11 

Other /unknown 

9 

This  study  found  that  despite 
the  existence  of  standards,  none  of 
the  12  states  strictly  follows  a 
standard.   Therefore,  a  compelling 
incentive  is  necessary  for  uniform 
standards  to  be  adopted  across  the 
nation.   Otherwise  a  broad  and 
continual  education  program  will  be 
required  and  may  still  be 
ineffective  in  achieving  conformity 
to  national  standards.   At  a 
minimum,  federal  guidance  and 
support  will  be  necessary  to 
encourage  the  acceptance  of 
standards. 
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THE  BALTIMORE  CITY/MARYLAND  STATE  SEXUALLY  TRANSMITTED  DISEASE  REGISTRY: 
IMPROVING  DISEASE  SURVEILLANCE  THROUGH  COMPUTERIZED  REPORTING 

Beth  Erickson,  Light  Industries 
Wayne  Brathwaite 


This  presentation  describes  a  collabora- 
tive effort  between  the  Baltimore  City  Health 
Department  (BCHD) ,  the  Maryland  State  Health  De- 
partment, and  Light  Industries,  a  computer  con- 
sulting firm  based  in  Millersville,  Maryland,  to 
develop  a  shared,  computerized  Sexually  Trans- 
mitted Disease  Registry.  Wayne  Brathwaite,  the 
co-author  of  this  presentation,  is  the  Manager 
of  the  STD  Program  in  Baltimore  City. 

We  present  the  Registry  as  a  model  of  a 
public/private  collaboration.  We  feel  that  the 
way  we've  worked  together  is  different  from  the 
typical  client/vendor  relationship  where  a 
client  contracts  out  to  a  vendor  for  a  specific, 
defined  task,  and  when  that  task  is  finished,  so 
is  the  relationship  and  the  commitment  to  see  it 
succeed.  Rather,  what  has  developed  between  us 
over  the  course  of  this  and  previous  projects  is 
a  deep  level  of  commitment  to  some  long-term 
computerization  goals.  There  is  a  mutual  re- 
spect for  the  talent  and  expertise  among  us  that 
has  created  a  synergistic  effect.  The  result  of 
this  synergy  has  been  a  series  of  projects  which 
we  think  are  more  successful  and  effective  than 
would  have  resulted  out  of  a  typical 
client/vendor  relationship. 

The  objectives  of  this  presentation  are 
to:  (1)  Provide  background  on  why  the  Registry  is 
needed,  the  benefits  of  the  registry,  and  how 
the  project  collaborations  came  about, 
(2)  Describe  the  methods  used  to  design  and  im- 
plement the  Registry,  (3) Describe  the  results  of 
the  design  process,  and  (4) Provide  some  observa- 
tions and  recommendations. 

BACKGROUND 

Why  an  STD  Registry  is  needed 


In  addition,  previous  test  and  treatment 
status  plays  a  role  in  determining  whether  to 
initiate  disease  investigation.  This  is  the 
process  during  which  Disease  Intervention  Staff 
(DIS)  go  into  the  field  to:  1) Verify  treatment, 
2) Coordinate  treatment  for  those  who  haven't  re- 
ceived it,  and  3) Conduct  contact  investigations 
with  partners  of  the  case. 

There  are  about  20,000  to  30,000  reports 
of  positive  syphilis  in  Baltimore  City  each 
year.  After  a  review  of  the  Registry's  histori- 
cal records,  only  about  20-30%  of  these  reports 
are  determined  to  need  field  investigation.  In 
this  way,  the  Registry  aids  in  making  the  most 
effective  use  of  limited  personnel  resources. 

The  manual  Registry  was  developed  in  1960 
and  until  1993,  when  the  computerized  Registry 
was  implemented,  the  Baltimore  STD  program  main- 
tained reports  of  positive  syphilis  on  5"  by  8" 
index  cards  which  were  stored  in  an  electric 
cardveyor . 

Benefits  of  Computerizing  the  Registry 

The  following  benefits  have  been  achieved 
with  computerization  of  the  Registry: 

1.  Simultaneous,  multi-user  access.  With  the 
manual  system,  only  one  staff  member  could  use 
the  cardveyor  at  a  time. 


2.  Faster  retrieval  of  information.  Using  the 
manual  system,  it  took  time  to  locate  the  cor- 
rect cardveyor  shelf,  and  within  it,  the  right 
card.  With  the  new  system,  a  record  is  located 
with  several  keystrokes. 


While  the  computerized  STD  Registry  col- 
lects information  on  syphilis,  gonorrhea  and 
chlamydia,  the  impetus  for  the  original,  manual 
Registry  was  the  need  for  information  about 
syphilis.  Syphilis  is  the  type  of  disease  where 
a  positive  test  does  not  necessarily  represent 
current  infection.  Therefore,  information  about 
previous  test  results  and  treatments  is  neces- 
sary to  determine  how  to  respond  to  the  current 
positive  test. 


3.  Permanent  record  location.  With  the  manual 
system,  once  a  card  was  misfiled,  it  (and  the 
information  on  it)  was  lost  indefinitely.  Com- 
puterized records  can't  be  misfiled. 

4.  Increased  accuracy.  With  the  manual  system 
there  was  no  routine  in  place  to  detect  data 
recording  errors.  The  computerized  system  pro- 
vides for  automatic  validation  of  data  as  it's 
entered . 
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5.  Increased  legibility.  Illegible  handwriting 
and  fading  ink  made  some  of  the  morbidity  cards 
difficult  to  read.  This  problem  is  eliminated 
with  electronic  display. 


1.  Like  the  City,  the  State  also  maintained  a 
manual  STD  Registry.  Computerization  of  their 
disease  reports  would  provide  the  benefits  just 
discussed. 


6.  Standard  data  format.  With  the  manual  sys- 
tem, data  could  be  entered  on  the  index  cards  in 
a  haphazard  format.  Now,  all  users  enter  data 
using  the  same  screens,  which  standardizes  the 
order  and  format  in  which  data  are  recorded. 


7.  Redundant  data  storage.  There  was  no  way  to 
create  a  back-up  of  the  card  system.  In  the 
event  of  a  fire,  for  instance,  the  Registry- 
would  have  ceased  to  exist.  Now,  the  Registry 
is  archived  daily  and  in  the  event  of  a  system 
failure  or  other  data  loss,  the  Registry 
datafiles  can  be  restored,  and  the  system  would 
be  up  and  running  in  a  few  hours. 


8.  Ease  of  merging  records.  In  the  manual 
system,  subsequent  reports  of  infection  were  ei- 
ther listed  on  the  previous  card,  or  a  new  card 
was  stapled  to  the  existing  card.  With  the 
computerized  Registry,  subsequent  disease  re- 
ports are  automatically  linked  to  the  previous 
records  of  the  case,  resulting  in  a  cumulative 
electronic  record  of  disease  status  over  time. 


2.  The  City  and  State  had  already  established 
a  collaborative  relationship  whereby  they  shared 
information  about  tests  reported  in  each  other's 
jurisdictions . 


3.  The  establishment  of  a  City/State  STD  Reg- 
istry is  consistent  with  Healthy  People  2000 
objectives  related  to  the  development  of 
surveillance  and  data  systems  across  State  and 
local  agencies. 

Both  the  City  and  the  State  obtained  fund- 
ing for  the  Registry  from  the  Centers  for  Dis- 
ease Control  and  Prevention.  System  design  re- 
sponsibility was  divided  based  on  the  amount  of 
money  each  party  could  contribute  to  the  pro- 
ject. Since  Baltimore  City  had  more  funds,  they 
were  responsible  for  the  design  and  implementa- 
tion of  the  core  system  modules.  The  State  is 
responsible  for  the  design  of  two  Disease  Inves- 
tigation modules. 

METHODS 


9.  Decreased  data  entry  time.  In  the  new  sys- 
tem, demographic  data  doesn't  need  to  be  entered 
with  each  new  report.  Previous  data  can  be 
edited  for  the  new  record,  and  the  old  data 
remains  accessible  to  the  user. 


10.  Increased  capability  for  data  analysis. 
Given  the  scope  of  the  data  collected  and  the 
volume  of  records  entered  into  the  system  each 
year,  the  possibilities  for  data  analysis  are 
unlimited,  for  instance,  to  produce  statistical 
reports,  to  evaluate  programs  such  as  STD 
surveillance  and  partner  notification,  to  answer 
research  questions  and  to  write  publications. 
The  data  also  provide  a  compelling  addition  to 
grant  applications. 


Benefits  of  a  City/State  collaboration 

There  were  several  reasons  why  the  City 
sought  collaboration  with  the  State  STD  Program 
which  is  directed  by  Kim  Seechuck: 


The  process  by  which  the  concept  of  the 
Registry  was  developed  into  the  system  that's  in 
use  today  was  accomplished  through  the  following 
steps : 

The  first  step  was  to  identify  the  techni- 
cal resource.  Since  1988,  the  BCHD  Sexually 
Transmitted  Disease  Program  had  worked  success- 
fully with  Light  Industries  on  several  projects, 
among  them  a  clinical  and  laboratory  information 
system  for  the  City's  two  STD  clinics.  During 
those  collaborations.  Light  Industries  staff  had 
acquired  considerable  knowledge  about  the  field 
of  public  health  and  STDs  in  particular,  and 
were  well  acquainted  with  how  the  STD  program 
functioned. 

Next,  specifications  were  developed  by 
Light  Industries.  This  process  began  when  the 
STD  Program  staff  provided  Light  Industries  with 
an  outline  of  how  the  Registry  should  work.  The 
systems  engineers  and  programmers  at  Light  In- 
dustries reviewed  the  material,  asked  questions, 
were  provided  clarification  by  the  Health  De- 
partment staff,  and  at  the  end  of  this  process, 
which  was  repeated  over  the  period  of  a  few 
months  and  over  the  course  of  several  meetings, 
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Light  Industries  synthesized  all  the  information 
and  developed  software  specifications.  The 
specifications  outlined  in  detail  what  the  sys- 
tem would  do  and  how  it  would  look  to  the  user. 

Upon  review  and  acceptance  of  the  specifi- 
cations by  the  BCHD  STD  Program  Manager,  the 
software  code  was  written  by  Light  Industries. 

Next,  the  software  was  installed,  and 
there  followed  a  period  during  which  coding  mod- 
ifications and  problems  resulting  from  concep- 
tual design  were  identified  and  resolved. 

Finally,  the  revised  code  was  installed 
and  at  that  point,  migration  from  the  old  manual 
system,  to  the  new  computerized  system  began. 

Since  then,  the  Registry  has  been  undergo- 
ing a  continual  process  of  enhancement.  Users 
identify  new  ways  that  the  system  can  help  them 
do  a  certain  task,  and  a  new  module  is  added  to 
the  basic  software  platform  that  was  installed 
several  years  ago.  Nonetheless,  the  development 
steps  outlined  here  are  followed,  no  matter  what 
the  scope  of  the  module  being  developed. 

RESULTS 


The  diagram  below  provides  an  overview  of 
the  design  and  function  of  the  Registry.  It  is 
maintained  on  a  local  area  network  at  the  admin- 


istrative office  of  the  Baltimore  City  STD  Pro- 
gram. The  Maryland  State  STD  Program  has  access 
via  a  remote  telecommunications  link.  Reports 
of  STDs  generated  at  the  State  Lab  are  transmit- 
ted to  the  Registry  via  diskette. 

Much  of  the  STD  testing  at  the  City's  two 
STD  clinics  and  two  Family  Planning  Clinics  are 
done  at  two  off -site  labs  (the  City  Lab  and  a 
research  lab  at  Johns  Hopkins  University) .  Re- 
quests for  STD  testing  from  these  clinics  are 
submitted  to  the  labs  via  a  Message  Handling 
System  (MHS)  data  transfer  protocol.  Test  re- 
sults are  transmitted  back  to  the  clinics  via 
the  MHS.  Upon  receipt  of  test  results,  the 
clinics,  i''i  turn,  transmit  the  disease  report, 
along  with  the  associated  patient  demographic 
data,  to  the  Registry  via  an  MHS  protocol. 

Five  City  School -Based  Clinics  can  submit 
results  to  the  Registry  via  diskette.  Written 
reports  from  other  test  sites  are  manually  en- 
tered into  the  Registry  upon  receipt  at  the  City 
or  State  STD  Program  offices. 

An  optical  disk  stores  images  of  the  STD 
morbidity  reports  from  the  cardveyor.  These 
images  are  linked  to  subsequent  electronic  dis- 
ease reports. 


The  Registry  provides  the  following  capa- 
bilities: 
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1 .  It  provides  for  quick  access  to  case  and 
contact  records  using  one  of  several  search 
methods. 


2.  It  captures  demographic  information  of  per- 
sons tested  for  syphilis,  gonorrhea  or  chlamydia 
and  allows  for  free  text  data  entry  via  an  auto- 
mated notepad  function. 

3 .  It  collects  information  about  test  results . 


4.  It  collects  information  about  the  type  of 
treatment  prescribed.  A  pull -down  menu  allows 
the  user  to  select  from  a  list  of  valid  treat- 
ment choices.  Treatments  can  be  marked  as  ade- 
quate or  inadequate  by  STD  program  staff,  allow- 
ing for  analysis  of  adherence  to  standard  treat- 
ment protocols  by  provider -type. 

5.  Information  about  the  Disease  Investigation 
process  is  collected,  for  instance,  whether  a 
field  investigation  was  initiated  and  if  the 
case  was  located  and  interviewed.  Information 
about  stage  of  disease  and  case  disposition  is 
also  collected. 


6.  Information  about  contacts  of  the  index  case 
can  be  entered  into  the  system,  for  instance 
basic  demographic  and  locating  information,  ex- 
posure information  and  results  of  the  Contact 
investigation.  The  information  collected  in 
this  module  is  very  powerful.  Due  to  the  fact 
that  patient  records  are  linked  to  contact 
records,  who  may  become  cases  themselves  if  they 
test  positive,  with  contacts  of  their  own,  this 
module  provides  valuable  information  for  the 
identification  and  analysis  of  disease  clusters. 

7.  Historical  morbidity  cards  from  the  card- 
veyor  are  scanned  into  the  system  using  an  opti- 
cal scanner.  The  images  are  linked  to  an  iden- 
tifying electronic  record  which  contains  just  a 
few  demographic  fields.  If  a  subsequent  test  is 
reported  for  an  individual  with  an  archived  im- 
age in  the  system,  the  new  electronic  report  is 
linked  to  the  image.  A  notation  appears  on  the 
demographic  screen  of  the  new  record  that  an 
image  exists  for  that  person.  The  image  can  be 
displayed  on  the  screen  by  pressing  a  hot  key. 


8.  The  system  imports  electronic  records  from 
labs  which  conduct  STD  testing.  Using  a  pre- 
defined set  of  matching  criteria,  the  system 


displays  potential  matches  between  the  imported 
(new)  disease  reports  and  existing  data.  The 
user  has  the  option  of  viewing  the  match  crite- 
ria for  each  record  before  selecting  the  best 
match  set. 


9.  The  Registry  contains  some  standard  system 
reports  (disease  status  by  race,  sex  and  age, 
for  instance) .  In  addition,  authorized  users 
have  access  to  an  off-the-shelf  report  writing 
software  program,  and  can  generate  ad  hoc  re- 
ports via  access  to  the  Registry  databases. 

CONCLUSION 

The  STD  Registry  provides  a  model  of  how 
public  and  private  sectors  can  collaborate  to 
respond  to  national  health  objectives. 

First,  through  automated  reporting  and 
multi-user  access  to  morbidity  information,  the 
Registry  addresses  national  objectives  to  in- 
crease the  proportion  of  providers  who  correctly 
manage  cases  of  STDs.  Currently,  work  is  under- 
way to  expand  on-line  access  to  the  Registry. 
By  the  Fall  of  1995,  clinicians  at  the  City's 
two  STD  clinics  will  be  able  to  access  the  Reg- 
istry databases  electronically  from  the  computer 
workstations  in  the  exam  rooms.  In  addition, 
we've  applied  for  funding  to  develop  a  method  by 
which  authorized  private  providers  in  the  commu- 
nity could  access  the  Registry  from  their  prac- 
tices . 

Secondly,  the  city/state  collaboration 
provides  a  model  system  for  the  transfer  of 
health  information  across  geopolitical  bound- 
aries -  one  of  the  Healthy  People  2000  surveil- 
lance and  data  systems  objectives. 

Lastly,  because  the  Registry  databases 
provide  a  vast  wealth  of  information  for  analy- 
sis and  research,  the  Registry  responds  to 
Healthy  People  2000  objectives  for  the  periodic 
analysis  and  publication  of  data  to  measure 
progress  toward  national  health  objectives. 

Obstacles 

An  obstacle  encountered  in  the  city/state 
collaborative  relationship  was  the  result  of 
dissimilar  administrative  goals.  Both  programs 
have  similar  reporting  responsibilities,  but  the 
City  STD  Program  has  an  additional  responsibil- 
ity to  provide  clinical  services  and  disease 
field  investigation.  Since  the  City  was  respon- 
sible for  the  design  of  the  Registry's  STD  re- 
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port  module,  data  elements  were  included  which 
would  facilitate  their  clinical  management  and 
field  investigation  responsibilities. 

After  using  the  system  for  some  time,  the 
State  STD  Program  decided  that  a  condensed  ver- 
sion of  the  STD  report  module  would  better  suit 
their  administrative  needs.  They  worked  with 
Light  Industries  to  modify  this  section  by 
deleting  the  clinical  and  disease  investigation 
data  elements.  This  solution  solved  the  prob- 
lem, however  because  there  are  now  two  versions 
of  the  software  in  operation  simultaneously,  the 
complexity  of  system  maintenance  and  modifica- 
tion has  increased. 

The  second  and  most  difficult  obstacle 
we've  encountered  has  to  do  with  fiscal  issues. 
With  a  project  of  this  nature  one  might  expect 
that  there  would  be  problems  getting  people  to 
work  together  or  problems  with  hardware  and 
software  design  or  implementation.  But  that 
wasn ' t  the  case . 

The  first  fiscal  obstacle  had  to  do  with 
obtaining  funds.  This  project  began  in  1992  and 
is  still  in  progress.  After  all  is  said  and 
done,  this  will  probably  be  a  five  year  project, 
but  what  we've  had  to  do  is  develop  it  one  year 
at  a  time,  within  the  confines  of  fiscal  funding 
cycles.  Our  belief  is  that  if  we'd  been  able  to 
obtain  a  longer  funding  commitment,  thereby  pro- 
viding a  broader  horizon  for  project  planning, 
the  result  may  have  been  a  more  efficient  use  of 
resources ,  and  an  even  more  successful  pro j ect . 

The  second  obstacle  has  to  do  with  spend- 
ing funds.  We've  found,  primarily  on  the  local 
level,  that  the  bureaucratic  purchasing  and  au- 
diting process  isn't  structured  to  work  well 
with  long-term  development  projects  where  tech- 
nology is  a  significant  issue.  The  people  who 
have  the  authority  to  approve  or  disapprove 
computer- related  technical  expenditures,  may  not 
have  the  expertise  to  evaluate  the  fiscal  sound- 
ness of  those  expenditures.  If  the  bureaucracy 
doesn't  support  or  understand  the  technical  con- 
cept, and  focuses  on  the  minutiae  of  project 
expenditures,  then  what  results  is  that  innova- 
tive technical  design  may  be  compromised  in  the 
vendor's  need  to  fit  an  inflexible  fiscal  model. 


Recorrmenda  tions 


ence  in  the  registration  packet.  The  report 
describes  the  National  Information  Infrastruc- 
ture (Nil)  initiative  and  presents  the  results 
of  an  April,  1995  conference.  One  of  the  goals 
of  the  conference  was  for  leaders  of  the  Na- 
tional Information  Infrastructure  initiative  to 
come  together  to  discuss  barriers  to  using  Nil 
technologies  in  public  health.  One  of  the  five 
obstacles  identified  was  "...organizational  and 
financing  issues  that  make  it  difficult  to  inte- 
grate information  systems  or  bring  potential 
partners  together."  While  specific  examples  of 
these  obstacles  were  not  presented,  the  state- 
ment suggests  that  the  fiscal  and  organizational 
problems  we've  encountered  in  this  project  are 
not  uncommon  to  others  who  collaborate  to  de- 
velop information  systems. 

In  order  to  feel  comfortable  with  allocat- 
ing fiscal  resources  for  longer-term  computeri- 
zation projects,  we  suggest  that  funding  for 
projects  be  evaluated  after  thoughtful  consider- 
ation of  the  long-term  benefits  of  the  project, 
relative  to  the  short-term  costs  of  implementing 
it.  We  recognize  the  need  for  funding  agencies 
to  monitor  the  use  of  public  funds,  but  suggest 
investigation  of  other  funding  approaches  that 
will  provide  the  fiscal  commitment  to  develop 
long-term  computerization  projects,  while  still 
maintaining  fiscal  accountability. 

With  regard  to  the  second  obstacle  - 
spending  funds,  we  suggest  that  some  thought  be 
given  to  developing  financial  accounting  models 
on  the  local  level  which  are  more  conducive  to 
facilitating  complicated,  technical  projects  be- 
tween public  and  private  entities. 

One  of  the  recommendations  that  came  out 
of  the  April,  1995  Nil  Conference  was  for  state 
and  local  public  health  agencies  to  "Actively 
seek  out  partnerships  and  funding  opportunities 
with  health  care  organizations,  medical  infor- 
matics groups,  and  the  commercial  sector." 

Before  this  can  occur  successfully,  how- 
ever, we  need  to  define  what  it  means  to  be  a 
partner  in  these  types  of  collaborative  rela- 
tionships which  are  encouraged  on  a  national 
level,  and  develop  fiscal  systems  on  the  local 
level  which  can  be  responsive  to  changes  in  the 
way  the  public  and  private  sectors  may  work  to- 
gether in  the  future. 


Of  relevance  to  the  discussion  of  fiscal 
obstacles  we've  encountered  in  this  project  is 
a  U.S.  Public  Health  Service  Report  (R.D. 
Lasker,  B.L.  Humphreys,  W.R.  Brathwaite)  which 
was  distributed  to  participants  of  this  confer- 
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DEVELOPMENT      OP      A      COMMUNITY-WIDE      CLINICAL      DATA      BASE      FOR 
CHILDREN'S      PRIMARY      CARE      WITH      PUBLIC      HEALTH      UTILITY 

Larry   Deutsch,    Children's   Health   Network 
Georgine   Burke,    Paul   Gionfriddo 


ABSTRACT 

A  wasteful  schism  exists  between  practices 
of  record  keeping  in  clinical  medicine,  and 
methods  of  data  collection  for  fiscal  and  public 
health  functions.  Within  each  realm, 
fragmentation  and  duplication  are  rife.   Wide 
community  acceptance  and  sharing  of  a  core  data 
set  requires  consensus  among  diverse,  perhaps 
competing,  interests.  The  value  of  health 
information  is  enhanced  by  the  ready 
availability  of  uniform  data  in  varied  contexts. 


areas,  information  access  is  characterized  by  a 
lack  of  continuity  and  coordination  among 
providers.  In  these  areas,  a  local  information 
infra-structure  and  a  patient-centered  system  of 
primary  care  are  missing.  Individual  and 
reference  data  are  often  lacking.  Decision- 
making and  follow  through  are  hampered,  with 
limited  involvement  of  patients  in  planning 
their  care,  and  insufficient  aggregate  data  for 
cost  analysis,  outcome  research,  community 
health  planning,  and  other  purposes. 


We  describe  a  process  to  collect  primary 
data  on  children's  health  status  for 
surveillance,  planning,  and  reform  efforts 
through  design  and  implementation  of  a 
comprehensive  multifunctional  clinically-driven 
data  base  among  regional  providers.  In  this 
Children's  Health  Network,  a  system  has  been 
developed  for  computerized  linkages  among 
heterogeneous  providers  serving  inner-city 
families  for  whom  multiple-site  use  is  common. 
Its  priority  is  the  quality  of  primary  care  for 
this  population,  emphasizing  integrated  service 
and  information  in  context  of  family  and 
community,  in  accordance  with  a  recent  Institute 
of  Medicine  (IOM)  redefinition. 
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The  core  data  set  is  an  enhancement  of  the 
National  Committee  on  Vital  and  Health 
Statistics  (NCVHS)  definition  of  a  Uniform 
Ambulatory  Care  Data  Set  (UACDS) ,  enlarged  for 
application  to  infants'  and  children's  services, 
computerized  with  open-architecture  design  and 
modular  capacity  to  permit  multi-institutional 
exchange  and  to  address  special  needs  of 
individuals  and  subpopulations . 

The  data  set  is  being  developed  to  serve 
clinical  and  public  health  needs.  A  community- 
consensus  process  is  being  used  to  address  the 
following  data  system  needs:   a)  continuity  and 
information-sharing  across  sites  for  high- 
quality  clinical  care,   b)  a  value-adding  format 
with  intelligent  features,  guidelines,  and  on- 
line services,  c)  new  benefits  for  families  such 
as  prompts  and  parent-held  native-language 
records,  d)  testing  of  current  confidentiality 
requirements  and  security  techniques  for 
electronic  data  sharing,  e)  automation  of 
functions  such  as  report  generation,  data 
analysis,  and  billing  to  achieve  economies  for 
providers  and  public  agencies,  and  f) 
accumulation  of  longitudinal  primary  data  for 
outcomes  research  and  public  health  planning. 


INTRODUCTION 

Health  care  is  at  its  best  when  both  the 
practitioner  and  patient  are  well-informed. 
However,  in  many  central  urban  and  remote  rural 


A  Children's  Health  Network  has  been 
designed  to  extend  current  information 
technology  to  these  underserved  areas.   Our 
approach  to  improving  quality  of  individual  care 
and  controlling  costs  emphasizes  use  of 
computerized  clinical  information  networks  for 
better  decision  making  and  continuity,  and, 
secondarily,  for  data  aggregation  for  financial, 
research,  and  public  health  planning.  This 
patient-centered  approach  is  in  distinction  to 
a)  data  systems  designed  primarily  for  billing 
and  administrative  functions,  and  b)  to  cost- 
control  efforts  which  rely  on  fiscal  and 
managerial  ("gatekeeper")  mechanisms.  A  uniform 
data  base  among  sites  serving  the  same 
population  will  answer  several  clinical  and 
public  health  needs. 


INCONSISTENT   CARE, 


POOR   DATA 


Too  often,  doctors  and  other  providers 
labor  with  knowledge  and  dedication  to  improve 
health  status  at  individual,  institutional,  or 
programmatic  levels,  but  without  broader 
awareness  of  how  the  welfare  of  an  individual  or 
community  is  affected  by  certain  decisions  or 
interventions . 

How  can  we  bring  this  systematic 
perspective  to  improve  health  conditions  of 
infants  and  children,  which  in  many  urban  and 
rural  areas  of  the  U.S.  are  far  from  optimal, 
and  below  that  of  many  other  countries?  It  is 
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clear  by  now  that  the  presence  of  technology 
itself  is  not  the  answer.  Deplorable  and 
preventable  health  conditions  are  found  within 
steps  of  our  cities'  technically-advanced 
medical  centers.1 

In  inner  cities  and  remote  rural  areas, 
well-publicized  preventable  conditions  include 
lead  toxicity,  injuries,  and  certain  behavioral 
disorders  and  infectious  diseases.  These  issues 
are  best  addressed  through  good  primary  health 
care,  as  well  as  amelioration  of  community 
environmental  and  socioeconomic  factors  which 
contribute  to  these  ills.   The  Institute  of 
Medicine  has  recently  re-defined  primary  care  as 
"the  provision   of  integrated 
[comprehensive,    coordinated,    and 
continuous] ,    accessible  health   care 
services  by  clinicians   who  are 
accountable   for  addressing  a   large 
majority  of  personal   health   care 
needs,    developing  a   sustained 
partnership  with  patients,    and 
practicing  in    the   context   of  family 
and  community. "   2 

In  contrast,  the  experience  of  clinicians 
and  families  shows  that  health  services  often 
lack  the  elements  of  integration  called  for  by 
the  IOM,  particularly  the  continuity  of  care  and 
coordination  of  services  required  for  adequate 
non-redundant  evaluation,  treatment,  and  follow- 
through.  These  failures  are  apparent  in 
impoverished  urban  areas  where  the  patient 
population  uses  multiple  facilities  -  despite 
strong  efforts  to  establish  "medical  homes."  In 
Hartford,  Connecticut,  for  example,  the 
population  is   1)  mobile:  61%  of  housing  units 
have  been  occupied  by  current  residents  for 
under  5  years;  2)    multicultural:  a  majority  of 
school  children  are  of  minority  background,  with 
a  Latino  plurality;  3)  poor:  68%  of  the  school 
population  is  Medicaid-eligible;  and  4)  have 
limited  or  inconsistent  access:  periods  with  or 
without  employment  and  varying  or  no  insurance 
coverage . 

Ambulatory  care  facilities  include 
community  health  centers,  hospital  out-patient 
departments  (OPDs),  school-based  clinics, 
visiting  nurse  associations,  and  a  few  private 
practices.  Among  these  facilities,  children 
often  lack  a  regular  source  of  ambulatory  care 
(RSAC) ,  more  so  for  poor  than  non-poor  (15%  vs 
8%)  in  national  survey  data.3   Those  with  RSAC 
made  80%  timely  routine  care  visits,  versus  48% 
for  those  without  one.  Moreover,  poor  children 
in  the  national  survey  more  often  (17%  vs  6%) 
received  sick  care  at  a  site  different  from 
routine  care,  a  discontinuity  particularly  for 
those  with  access  to  a  community  health  center 
rather  than  a  physician's  office  (40  vs  4%).  For 
Hartford,  67%  of  those  at  an  off-hours  OPD 
clinic  usually  attended  a  different  facility  for 
care,  including  22%  indicating  "none"  for  RSAC.4 

These  community  sites  operate  with 
different  hours,  changing  staffing,  incompatible 
data  systems,  and  conflicting  priorities. 
Although  these  facilities -serve  largely  one 


population,  they  have  generally  been  unable, 
through  lack  of  planning  and  financing,  to 
organize  and  support  comprehensive  up-to-date 
information  systems  among  themselves. 
Information  transfer  under  actual  conditions  is 
primitive,  with  illegible  carbons,  late  letters, 
incomplete  phone  messages,  etc.  This  concretely 
impairs  the  thoroughness  and  effectiveness  of 
dedicated  and  well-intentioned  health  providers. 

Furthermore,  data  are  institution-  rather 
than  patient-centered.  Documentation  is  often 
non-uniform,  serves  primarily  a  billing 
function,  and  resides  in  distinct,  single- 
function  data  bases  (e.g.,  for  lead  monitoring, 
immunization  status,  nutrition  or  social  service 
programs,  specific  disease  entities)  .  Without 
expensive  and  time-consuming  labor,  aggregation 
of  the  same  data  for  multiple  functions  - 
longitudinal  tracking  for  outcome  research, 
utilization  and  cost  analysis,  public  health 
program  planning,  and  policy  formation  -  is 
virtually  impossible.  In  summary,  infants  and 
children  are  subject  to  well-understood  health 
hazards  and  suffer  impairment  due  to  our 
unwillingness  or  inability  to  distribute  and 
apply  systematically  the  technical  tools 
currently  at  hand. 


PRIMARY   CARE,   CLINICAL 
SYSTEMS,   AND  COSTS 


INFORMATION 


Lack  of  adequate  systems  for  primary  care 
in  the  United  States  leads  not  only  to 
suboptimal  health  status  reflected  in  local  and 
national  indices,  but  also  to  poor  cost 
effectiveness.  Cost  savings  related  to 
particular  improvements  in  primary  care  have 
been  shown;  good  longitudinality  (consistent 
caregiver,  site,  and  record),  for  example 
reduces  hospitalization,  preventable  illness, 
and  overall  costs  of  care.5  Great  savings  result 
from  regular  use  of  community  health  centers  and 
school-based  clinics  rather  than  emergency 
departments . 

Cost  benefits  have  also  been  shown  with 
computerization  of  health  records.  In  its 
important  volume  The     Computer-baaed    Patient 
Record:      an     essential      technology     for 
health    care,6   the  IOM  records  numerous  areas, 
such  as  unnecessary  or  redundant  laboratory 
testing,  medication  errors,  and  data  entry 
inefficiencies,  for  which  expenses  have  fallen 
using  modern  clinical  and  management  information 
systems.  Inclusion  of  on-line  library  reference 
services  also  reduces  costs  of  patient  care7. 
Demonstration  projects  with  clinical  information 
systems  operating  under  real  conditions  are 
needed  to  measure  cost-savings  (as  well  as 
improved  care  quality  and  health  outcomes)  among 
large  and  small  institutions  serving  community 
populations.2'6'8    This  system  approach  is 
distinct  from  a  focus  on  case  management, 
electronic  billing,  or  utilization  control. 
Research  in  linked  clinical  system  models  is 
pertinent  to  state  and  federal  agencies,  e.g.  in 
the  monitoring  of  Medicare  and  Medicaid  programs 
if  there  is  shift  to  managed  care,  and  to 
private  sector  alliances. 
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In  summary,  a  lack  of  communication  and 
coordination  frameworks  utilizing  current 
technology  impairs  health  status  and  elevates 
costs.  Creation  of  new  local  information 
infrastructures  linked  with  statewide  and 
national  services  will  address  some  of  these 
system  failures. 

METHODOLOGY 


give  health  providers  on-line  access  at 
each  encounter  to  summary  patient 
information,  including  demographic, 
medical  and  risk-factor  data;  to  assist 
decision-making  and  document  treatment 
plans;  and  to  permit  follow-through 
including  automated  prompts,  record 
transfers,  and  outreach  for  active  health 
assurance  and  prevention  programs; 


The  Children's  Health  Network  is  a  project 
conceived  by  numerous  individuals  and 
institutions  concerned  with  improving  the 
quality  of  health  services  for  infants  and 
children.  It  began  with  formation  of  a  multi- 
disciplinary  task  force  and  developed  with 
support  from  local  foundations  and  the  Maternal 
and  Child  Health  Bureau  (Health  Resources  and 
Services  Administration,  Public  Health  Service, 
Department  of  Health  and  Human  Services) .  The 
task  force  formulated  working  groups  addressing 
specific  areas: 

•  content  of  shared  uniform  clinical  data 
base  ; 

•  coordination  among  network  sites; 

•  design  of  overall  network  architecture  and 
software; 

•  legal  research  in  emerging  confidentiality 
standards  and  law  to  assure  privacy  of 
information  and  system  security,  and  to 
recommend  policies  and  procedures  for 
participating  sites; 

•  relationships  with  outside  institutions 
and  agencies  seeking  aggregated  data  for 
research  and  public  health  functions; 

•  evaluation  of  Network  conception, 
functioning,  utility,  and  user  (provider 
and  family)  satisfaction. 

The  network  design  developed  was  an  open- 
architecture  PC-based  distributed  client-server 
system  using  Windows  and  a  modern  SQL  relational 
data  base.9  Modular  elements  for  particular 
diagnostic  entities,  disease  states,  artificial 
intelligence,  graphical  capabilities,  and  report 
generation  may  be  added  depending  on  provider 
interests  and  needs. 

The  systsm  design  anticipates  flexible  data 
entry  techniques  and  multiple  functions  for  its 
demographic  (enrollment -based)  and 
comprehensive  clinical   (encounter-based) 
information,  expanding  the  proposed  uniform 
ambulatory  care  data  set  (UACDS)  with  fields  of 
pediatric  importance  (growth,  immunizations, 
dental  care,  development,  social  services)  and 
incorporating  standards  for  data  definitions  and 
exchange  among  heterogenous  institutions,  as 
they  emerge . 

For  single-  or  multiple-site  use,  the 
program  through  its  relational  data  base  and 
network  connectivity  is  preparing  to: 


•  accumulate  longitudinal  patient-level  data 
summarizing  conditions  and  interventions 
(medical  and  social)  for  medical  treatment 
effectiveness  studies  and  other  research; 

•  enable  providers  to  receive  from  and 
report  to  various  specific  data  bases, 
fiscal  organizations,  and  public  agencies. 

•  accommodate  new  roles  for  patient/family 
participation  in  health  care,  with 
personal  (hand-held)  records  in  native- 
language  translation  and  patient  education 
materials; 

•  allow  reporting  of  aggregate  data  for 
local  community  health  monitoring, 
including  small-area  analysis,  needs 
assessment,  surveillance  for  specific 
entities,  and  discovery  of  trends,  program 
assessment  and  resource  allocation,  public 
health  planning  and  policy  formation; 

•  meet  emerging  requirements  for  quality 
assurance  (QA) ,  and  for  utilization  and 
cost  monitoring  with  managed  care  or 
other  frameworks,  including  Medicaid 
(indigent)  populations. 

Pilot  sites  for  the  Children's  Health 
Network  project  include  a  paired  elementary 
school  health  clinic  and  nearby  pediatric 
ambulatory  care  center  in  a  major  urban  hospital 
in  Hartford,  CT.  Additional  linkages  are  planned 
(Fall,  1995)  for  a  community  health  center  and 
high  school  health  clinic  in  Hartford,  and 
different  clinics  and  practices  in  New  Haven  and 
other  cities. 

PUBLIC   HEALTH   SYSTEMS   AND   BARRIERS   TO 
COMMUNITY   INFORMATION  NETWORKS 

Several  barriers  have  been  identified  in 
applying  a  U.S.  National  Information 
Infrastructure  to  public  health.  These  include: 
the  need  to  protect  privacy  of  health  data;  the 
lack  of  national  data  collection  and  maintenance 
standards;  insufficient  understanding  of 
applications  of  Nil  technology  to  public  health; 
and  lack  of  experience  in  the  public  health 
workforce  with  Nil  technology. 

The  Network  has  benefitted  from  development 
by  the  NCVHS  of  its  core  data  set  (UACDS) .  In 
adopting  its  sixteen  fields  as  a  starting  point, 
the  Network  hopes  that  its  data  will  be 
comparable  to  others  around  the  country, 
enabling  researchers  and  policy  makers  to 
compare  different  populations.  By  creating 
additional  fields,  the  Network  has  sought  to 
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meet  the  needs  of  local  children's  clinicians 
and  institutions  to  have  more  complete  and 
portable  medical  records. 

Considerable  effort  has  been  devoted  to 
development  of  policies  and  protocols  to  protect 
the  confidentiality  of  patient-specific  data 
while  allowing  information  exchange  among 
clinicians.  Procedures  have  been  developed  for 
transfer  of  records  only  when  informed  consent 
from  parents  has  been  entered,  protection  of 
data  at  a  site  when  no  consent  has  been 
documented,  and  policies  for  aggregate  data  use 
for  public  health  purposes.10'11 


it  is  concretized  in  new  programs  through  the 
National  Library  of  Medicine,  Agency  for  Health 
Care  Policy  and  Research,  Office  of  Rural  Health 
Policy,  and  the  National  Telecommunications  and 
Information  Administration.12   Assistance  to 
these  community  institutions  should  emphasize 
patient-centered  computer-based  record  systems 
with  uniform  minimum  data  bases  and  security 
features.  Systems  should  allow  adaptability  for 
local  needs  and  reporting  requirements, 
integration  of  legacy  systems,  and  linkage  to 
local  and  regional  networks  which  in  turn  are 
joined  to  the  national  information 
infrastructure.  Policies  may  include: 


The  Network  has  worked  to  reduce  cost 
barriers  to  computer-based  records  by  designing 
a  distributed  system  using  low-cost  personal 
computers  with  Windows  and  a  common  data  base 
software.  With  ongoing  training  of  providers  and 
easy-to-use  formats  for  data  display  and  report 
generation,  system  users  may  become  familiar 
with  its  operation  in  a  short  time. 

Finally,  while  the  Network  was  designed  to 
improve  continuity  and  quality  of  care  through 
more  efficient  information  exchange,  it  is 
intended  that  data  residing  in  the  system  will 
provide  valuable  information  for  public  health 
activity,  including  outcome  research,  program 
evaluation,  and  policy  making.  In  building  such 
a  multifunctional  system,  it  has  benefitted  from 
Maternal  and  Child  Health  Bureau  (PHS)  support. 

POLICY   IMPLICATIONS 

With  the  IOM  criteria  for  primary  care  in 
mind,  a  traditional  model  is  the  old-style 
family  practitioner,  rooted  in  community, 
personally  familiar  with  individual  and  family, 
available  all  the  time  without  financial, 
language,  bureaucratic,  or  other  barriers, 
acting  as  patient  advocate  and  care  coordinator 
(rather  than  "manager"  or  "gatekeeper"),  able  to 
make  intelligent  use  of  additional  local  or 
distant  facilities  and  information  when 
warranted  and  receive  feedback  from  colleagues 
and  local  institutions  for  continuity  of  care. 
But  times  have  changed.  Current  models  of 
primary  and  preventive  health  practice  in  inner 
city  and  rural  areas  stress  community-oriented 
primary  care  (COPC)  and  support  for  community 
health  centers  and  school-linked  health 
services,  as  well  as  health  maintenance 
organizations  and  wider  fiscal-administrative 
networks  or  alliances  for  managed  care. 
Discussion  of  these  is  beyond  our  scope.  Some 
models  may  recognize  the  need  for  patient- 
centered  computer-based  records  and  improved 
aggregate  data. 

A  major  focus  of  policymakers'  attention  in 
these  times  must  be  proactive  extension  of 
current  and  new  information  technology  to 
smaller  community-based  primary  care  providers  - 
those  on  the  front  line  of  health  care  delivery, 
particularly  to  underserved  populations.  This  is 
in  keeping  with  concepts  presented  by  Vice 
President  Gore  to  ensure  access  for  all 
communities  to  the  information  superhighway;  and 


•  sustained  funding  of  smaller  primary  care 
institutions  including  community  health 
centers,  school-based  clinics,  and  home- 
visiting  services,  and  their  integration 
into  community  health  information  networks 
(CHINs)  or  other  regional  information 
infrastructures ; 

•  investment  in  a  variety  of  community-based 
demonstration  projects  which  model 
clinically-driven  information  systems 
yielding  multifunctional  aggregate  data; 
these  systems  are  distinct  from  -  neither 
exclusive  of  nor  equating  to  -  management 
information  systems  which  rely  on  fiscal 
and  managerial  ("gatekeeper")  mechanisms 
for  control  of  costs; 

•  for  all  primary  care  providers, 
availability  of  standard,  flexible, 
regularly-revised  public-domain 
software,  including  programs  to  meet 
burdensome  newly-emerging  requirements  for 
quality  monitoring,  fiscal  and  agency 
reporting,  etc. 

•  provision  and  maintenance  of  networking 
equipment  including  linkage  hardware, 
fiber  optic  or  other  secure  line 
connections,  connectivity  fees,  conversion 
software  to  accommodate  new  and  legacy 
systems,  and  stand-alone  versions  for 
remote  or  non-networked  institutions; 

•  expansion  through  local  networks  of  on- 
line cost-free  reference  services  through 
the  National  Library  of  Medicine  and 
regional  academic  medical  centers; 

•  on-going  technical  assistance  through 
state  or  federal  sources  to  the  many  sites 
lacking  expertise  (information 
departments)  internally,  especially  to 
meet  new  quality  monitoring  and  reporting 
requirements  and  adapt  to  technical 
advances . 

There  are  many  implications  of  comprehensive 
primary  care  systems  such  as  the  Children's 
Health  Network  that  relate  to  problems 
confronting  clinicians,  payers,  and  policy 
makers  in  the  1990s.  The  Network's  development 
and  findings  will  be  beneficial  to  others 
engaged  in  similar  efforts. 
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Introduction 

Disability  is  a  multifaceted  phenomenon. 
Health-related  limitations  can  occur  in  numerous 
roles  and  activities,  such  as  job,  personal  care, 
household  management,  socializing  with  friends, 
active  recreation,  and  passive  leisure.  There  are 
also  various  dimensions  of  those  limitations, 
such  as  degree  of  difficulty,  use  of  equipment  or 
personal  assistance,  pain  while  engaged  in  the 
activity,  and  satisfaction  with  performance. 
Faced  with  such  diversity  when  designing  surveys, 
researchers  select  certain  activities  and 
dimensions  that  seem  most  germane  for  the  age- 
gender  groups  studied  or  for  public  policy.  The 
most  common  choices  are  questions  about 
difficulty  or  assistance  in  performing  personal 
care  (ADL),  household  management  (IADL),  and  job 
activities.  Even  with  this  restricted  scope,  the 
number  of  disability  questions  in  surveys  has 
become  large,  posing  burdens  for  interviewers, 
respondents,  survey  analysts,  and  disability 
statistics  users.  And  pressures  continue  for 
still  further  expansion  of  disability  items. 

In  distinct  contrast  to  this  situation, 
there  has  been  movement  toward  parsimony  in 
measuring  morbidity.  Although  health  status  is 
also  multifaceted  (presence/absence  of  specific 
conditions,  severity,  duration,  etc.),  a  global 
item  to  summarize  it  is  routinely  included  in 
surveys:  self-rated  health.  Its  value  for 
prediction  of  dire  outcomes  such  as 
institutionalization  and  death  is  equal  to  or 
better  than  arrays  of  detailed  morbidity  items. 
The  item  is  brief  to  administer  and  has  good 
colloquial  merit  ("makes  sense"  to  respondents). 
In  short,  one  question  about  health  happens  to  be 
realistic,  comprehensive,  and  prescient. 

The  sharp  difference  between  survey 
approaches  to  disability  and  morbidity  is  the 
underlying  motivation  for  this  paper.  For  all  the 
interest  in  and  wide  use  of  a  global  morbidity 
item,  there  has  been  little  work  to  develop  and 
use  a  global  disability  item.  It  may  indeed  be 
possible  to  find  one  that  has  strong  analytic 
value  and  also  captures  the  real-world  experience 
of  disability  well.  If  so,  it  should  be  regularly 
included  in  health  surveys  either  as  a  companion 
to  detailed  disability  questions  (in  surveys  with 
extensive  focus  on  chronic  morbidity  and 
functioning)  or  by  itself  (in  surveys  with  sharp 
time  limits  or  brief  coverage  of  morbidity/ 
functioning) .  Alternatively,  it  may  be  possible 
to  reduce  the  number  of  detailed  items  about 
disability  by  dropping  some  activities  or 
dimensions,  with  little  loss  of  analytic  value. 
Either  approach — a  global  item  or  reduced  items — 
will  achieve  disability  parsimony,  but  the  global 
item  is  certainly  most  economical. 

This  paper  has  four  parts:  First,  we  provide 
some  background  for  thinking  about  disability 
measurement  and  the  issue  of  parsimony.  Second, 
we  present  empirical  results  from  three  projects; 
one  relates  chronic  conditions  to  a  great  number 
of  detailed  disability  items,  and  the  others 
relate  a  global  disability  item  to  detailed  ones 
and  to  self-rated  health.  Third,  from  the  results 
we  draw  conclusions  about  (a)  how  detailed 
disability  items  can  be  culled  and  (b)  whether 
global  disability  is  worth  adding  to  surveys. 
Lastly,  recommendations  are  offered  about  work 
that  can  be  done  with  existing  surveys  or  in 
small-scale  laboratory-based  studies  to  promote 
compact  questioning  of  disability  in  health 
surveys . 

Background  on  Global  and  Detailed  Disability 
Items 

Disability  refers  to  the  impacts  health 
problems  have  on  people's  social  functioning, 
that  is,  their  ability  to  perform  roles  and 
activities  (Pope  &  Tarlov,   1991;  Verbrugge  & 


Jette,  1994).  "Social  functioning"  includes  the 
whole  range  of  typical  and  personally-desired 
activities  an  individual  does,  ranging  from  the 
most  basic  and  universal  (such  as  eating  and 
dressing)  to  the  most  discretionary  and 
distinctive  (such  as  a  person's  favorite  hobby  or 
recreation) .  Disability  can  be  short-term  or 
long-term,  and  it  can  be  due  to  acute  or  chronic 
conditions.  Because  research  and  policy  interests 
are  typically  on  long-term  dysfunctions 
associated  with  chronic  conditions,  that  is  our 
focus  here. 

The  aim  of  a  global  disability  item  is  to 
measure  overall  social  functioning  briefly  but 
well.  It  must  refer  to  protracted,  health-related 
difficulties  in  a  large  span  of  activities.  The 
question  format  can  be  one  single  question,  a 
branch-and-stem  item  (main  question  plus  probes 
about  duration  and  health-relatedness)  ,  or  a 
small  set  of  questions  (short  ones  that  are 
combined  into  a  single  variable  during  analysis). 
These  formats  are  compact  in  the  questionnaire 
itself,  and  thus  brief  to  administer  and  easy  to 
analyze.1 

There  are  some  examples  of  global  disability 
items  in  contemporary  U.S.  and  Canadian  surveys 
(Verbrugge,  1994).  Most  have  been  created  through 
a  mixture  of  judgment  and  consultation,  copying 
from  prior  surveys,  and  pretesting.  Ideally, 
choices  should  be  based  also  on  empirical 
evidence  about  content  (what  aspects  of 
disability  the  item  covers)  and  analytic  value 
(relationships  to  predictors  or  outcomes).  Little 
evidence  of  that  sort  exists.  So  good 
craftsmanship  is  the  mainstay  for  designing 
items. 

What  sorts  of  methodological  work  can  help 
in  evaluating  global  disability  items?  There  are 
two  basic  approaches:  cognitive  and  statistical. 
Cognitive  approaches  are  well  suited  to  studying 
the  processes  that  respondents  use  to  think  about 
questions  and  come  up  with  answers.  These  studies 
are  usually  small-scale  and  often  laboratory 
based.  Statistical  approaches  are  used  on 
moderate-to  large-scale  data  sets  to  study 
multivariate  item  structure,  reliability,  and 
concurrent  and  predictive  validity.  Examples  of 
analyses  that  can  inform  us  about  global 
disability  are  (1)  relationships  between  a  global 
item  and  specific  disability  items,  to  determine 
the  global  item's  included  and  excluded  content, 
(2)  relationships  between  global  disability  and 
global  morbidity,  to  see  if  they  are  non- 
redundant,  (3)  models  relating  chronic  conditions 
and  global  disability,  to  assess  its  health- 
relatedness,  and  (4)  the  prediction  ability  of 
global  disability  by  itself  (apart  from  self- 
rated  health)  on  subsequent  outcomes.  In  general 
the  evidence  compares  and  contrasts  global 
disability  with  (a)  detailed  disability  items  and 
(b)  global  morbidity.  Ideally,  one  wants  a  global 
disability  item  to  have  good  coverage  of  detailed 
disabilities  (high  correlations  with  them)  and  be 
distinctive  from  self-rated  health  (moderate-to- 
low  correlation  and  strong  net  relationship  to 
outcomes) . 

The  notion  of  parsimony  is  also  relevant  for 
surveys  that  contain  detailed  disability  items, 
the  current  situation  for  contemporary  health 
surveys  (especially  of  older  persons).  The 
numerous  disability  questions  are  usually  about  a 
rather  narrow  set  of  activities  (ADLs,  IADLs, 
job;  also  physical  and  sensory  limitations),  with 
several  dimensions  for  each  (difficulty, 
equipment  assistance,  personal  assistance) . 
Parsimony  could  be  achieved  by  reducing  the 
number  of  detailed  items.  Statistical  approaches 
can  inform  us  on  this  issue;  for  example, 
analyses  of  (5)  relationships  between  chronic 
conditions  and  specific  disabilities,  to  assess 
whether  disabilities  have  similar  morbidity 
precursors,  (6)  how  detailed  items  predict 
prospective  dire  outcomes,  and  (7)  clustering  and 
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hierarchy  of  items,  assessing  if  any  given 
detailed  guestion  actually  represents  a  whole 
disability  profile.  Items  with  low  health- 
relatedness  or  low  prediction  can  be  considered 
for  elimination.  If  scaling  analyses  show  strong 
hierarchy,  then  an  economical  approach  to  asking 
about  disability  can  be  considered  (items  are 
ordered  according  to  the  scale,  questioning 
begins  somewhere  in  the  middle,  and  it  proceeds 
'up'  or  'down'  the  scale  until  a  yes  for 
disability  occurs). 

We  now  present  results  of  three  projects 
motivated  by  our  interest  in  parsimony.  One 
studies  the  health-relatedness  of  numerous 
detailed  disability  items  (#5  above).  The  second 
and  third  study  relationships  of  a  global 
disability  item  to  detailed  disabilities  (#1)  and 
self-rated  health  (#2).  The  second  project  also 
considers  the  global  item's  health-relatedness 
(#3).  We  make  conclusions  about  winnowing 
detailed  items  and  about  the  merits  of  a  global 
item.  The  next  three  sections  discuss  the 
projects,  one  at  a  time. 

Morbidity  Precursors  of  Detailed  Disabilities 
(Project  1:  AHEAD) 

Are  chronic  conditions  strongly  related  to 
presence  and  degree  of  disability,  or  only  weakly 
so?  Are  the  links  between  morbidity  and 
disability  distinctive  (different  chronic 
conditions  are  implicated  for  each  disability)  or 
nondistinctive  (the  same  conditions  come  into 
play  for  virtually  all  disabilities)?  The  answers 
will  indicate  the  health-relatedness  of 
dysfunctions  and  similarities  in  morbidity- 
disability  relationships. 

We  utilized  data  from  the  AHEAD  Survey 
(Asset  and  Health  Dynamics  of  the  Oldest-Old) 
Wave  1  (Merrill  &  Verbrugge,  1995).  AHEAD  is  a 
population-based  sample  of  U.S.  community- 
dwelling  persons  ages  70+  at  Wave  1  (1993-94); 
n=8,224.  The  questionnaire  has  information  on 
presence/absence  of  25  chronic  conditions,  22 
specific  disabilities  (ADLs,  IADLs,  physical 
limitations;2  with  various  dimensions:  degree  of 
difficulty,  use  of  assistance,  need  for 
assistance,  pain  when  doing  activity,  tiredness 
when  doing  it,  long  time  to  do  it),  and  9 
productive  activities.  The  disability  items  were 
used  as  is  and  also  in  aggregated  forms  (such  as 
"any  ADLs"  and  "sum  of  ADLs").  The  full  set  of 
chronic  conditions  (X)  were  related  to  each 
disability  outcome  (Y)  by  logistic  and  linear 
regressions,  controlling  for  age  and  gender. 

Descriptive  statistics  for  variables  and 
tables  with  results  are  in  Merrill  and  Verbrugge 
(1995).  Three  tables  most  pertinent  to  this 
article  are  available  on  request:  One 
illustrates  results  for  detailed  disability  (the 
table  is  for  physical  limitations),  the  next 
illustrates  results  for  aggregated  disability 
(the  table  is  for  sums),  and  the  last  lists 
chronic  conditions  that  always/almost  always  have 
significant  relationships  with  disability  items. 

1.  The  most  striking  result  is  that  the  same  8 
chronic  conditions  routinely  have  statistically 
significant  associations  with  the  many  disability 
items  (detailed  and  aggregated) .  They  are  stroke, 
diabetes,  arthritis,  hip  fracture,  urinary 
incontinence,  poor  vision,  frequent  pain,  and  a 
residual  "other  conditions".3  The  other  chronic 
conditions  are  related  to  certain  disabilities  or 
disability  domains,  but  not  consistently  across 
the  board. 

2.  R2s  are  generally  .10-. 20  for  specific  ADLs  and 
IADLs,  and  .20-. 30  for  specific  physical 
limitations.  Aggregated  variables  (such  as  any 
ADLs  and  sum  of  ADLs)  produce  higher  R2  than  their 
detailed  source  items,  the  increase  being  about 
.10.  Moreover,  more  chronic  conditions  have 
significant  relationships  with  these  aggregate 
items  than  with  the  detailed  ones. 

The  results  lead  to  two  conclusions:  (1) 
First,  there  is  plenty  of  redundancy  in  the 
health-relatedness  of  disability  items.  Thus,  if 
a  survey  needs  to  include  the  topic  of  disability 


but  does  not  really  need  disability  details,  then 
any  4-5  items  will  serve  that  purpose  adequately. 
The  most  sensible  choice  is  asking  one  dimension 
(such  as  difficulty)  for  several  diverse 
activities  (spanning  ADLs,  IADLs  and  physical 
limitations).  (2)  Second,  the  association  between 
morbidity  and  specific  disabilities  (R2)  is 
modest,  but  increases  notably  for  aggregated 
disability  variables.  Thus  for  analytic 
parsimony,  one  should  use  the  aggregates  and  skip 
the  detailed  items.  But  there  is  no  fieldwork 
parsimony  in  this  approach,  since  aggregate 
variables  depend  on  having  asked  the  plethora  of 
detailed  items  1  In  short,  the  AHEAD  analyses 
suggest  how  to  use  detailed  items  with  parsimony 
in  two  ways:  by  reducing  the  number  of  detailed 
items  placed  in  a  questionnaire  or,  if  that 
doesn't  happen,  by  reducing  the  number  of 
disability  variables  analyzed. 

Distinctive  Features  of  Global  Disability 
(Project  2:  HRS) 

Is  global  disability  related  to  all  specific 
disabilities,  or  to  some  specific  ones  far  more 
strongly  than  others?  Is  global  disability 
closely  related  to  global  morbidity  (self-rated 
health)  or  weakly?  How  health-related  is  global 
disability?  The  answers  will  indicate  how  well  a 
global  item  compasses  activity  domains,  whether 
it  is  really  something  different  from  global 
morbidity,  and  how  well  it  reflects  underlying 
health  problems. 

We  utilized  data  from  the  HRS  (Health  and 
Retirement  Survey)  Wave  1.'  HRS  is  a  population- 
based  sample  of  U.S.  community  dwellers  ages  51- 
61  in  1992,  plus  their  spouses;  n=12,654.  The 
questionnaire  has  information  on  disability 
(limitation  in  job,  housework,  or  other  activity, 
5  ADLs:  and  difficulty  doing  each,  5  physical 
limitations:  any  difficulty  doing  each;  no  IADLs 
asked)  and  health  status  (presence/absence  of  19 
specific  conditions,  self-rated  health). 

We  used  the  limitation  in  job/housework/ 
other  items  to  create  a  global  disability 
variable,  as  follows:  All  persons  are  asked  if 
they  have  an  impairment  or  health  problem  that 
limits  the  kind  or  amount  of  paid  work  they  can 
do.  The  subset  saying  no  are  asked  about  health- 
related  housework  limitations;  and  lastly,  the 
further  subset  saying  no  to  housework  are  asked 
about  health-related  "limitation  in  any  way  in 
activities".  Our  global  variable  is  dichotomous, 
scored  1  if  yes  to  any  of  the  three  items,  0 
otherwise.5  The  percentages  are  29.5%  disabled  and 
70.5%  non-disabled.  (The  building-block 
percentages  from  the  three  items  are:  21.5%  job 
limitation,  additional  3.5%  housework  limitation, 
additional  4.5%  other  limitation.)  The  other 
disability  items  were  used  as  is  and  also  in 
aggregated  forms  (such  as  "any  ADLs"  and  "sum  of 
ADLs" ) . 

For  the  first  part  of  the  analysis,  detailed 
disabilities  (X)  are  related  to  global  disability 
(Y)  in  logistic  regressions.  For  the  second 
part,  aggregate  and  global  disability  (X)  are 
related  to  self-rated  health  (Y)  in  logistic 
regressions.  Lastly,  we  are  able  to  look  again 
at  the  health-relatedness  of  disability,  this 
time  with  a  genuine  global  disability  item.  Age, 
gender  and  education  are  controlled  in  all 
regressions. 

Descriptive  statistics  for  variables  are 
available  on  request.  Four  tables  (referenced  at 
suitable  points  below)  are  included  with  this 
article. 

1.  How  are  detailed  disabilities  associated  with 
the  global  indicator  (TABLE  1)?  Three  models  were 
estimated:  with  both  physical  limitations  and 
ADLs  as  predictors  (Model  I),  just  ADLs  (Model 
II),  and  just  physical  limitations  (Model  III). 
On  their  own,  physical  limitations  are  strongly 
linked  with  global  disability  (Model  III).  Each 
item  (walk  blocks,  climb  stairs,  pull/push  large 
objects,  lift/carry  ten  pounds,  pick  up  dime)  has 
a  statistically  significant  coefficient  for  the 
total  sample  and  each  gender.  Similarly, 
difficulties  in  ADLs  (walk  across  room,  bathe, 
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transfer  in/out  of  bed,  dress;  but  not  eating) 
are  also  associated  with  global  disability  (Model 
II).  But  in  models  with  combined  predictors 
(Model  I),  the  ADL  coefficients  fade  in  size 
while  those  for  physical  limitations  remain 
essentially  as  large  as  before.  Log  likelihood 
values  show  this  relative  importance  as  well; 
ADLs  add  almost  nothing  to  the  prediction 
strength  of  physical  limitations  (comparing 
Models  III  and  I) . 

2.  The  relationship  of  disability  to  self-rated 
health  was  studied  next  (TABLE  2).  Self-rated 
health  is  scored  in  two  ways,  as  a  dichotomous 
variable  of  poor  vs.  other  responses ( fair,  good, 
very  good,  excellent)  and  a  dichotomous  variable 
of  poor/fair  vs.  good/very  good/excellent.  Only 
8.0%  of  this  middle-aged  sample  report  poor 
health;  14.3%  report  fair  health.  Results  show 
that  our  models  (with  straightforward  morbidity 
and  sociodemographic  predictors)  do  a  better  job 
predicting  fair/poor  health  than  poor  health — not 
surprising  given  the  rarity  and  thus  "unusual 
circumstances"  underlying  poor  health  at  these 
ages.  Of  the  three  disability  variables,  global 
disability  has  the  strongest  relationship  to 
self-rated  health;  this  is  seen  both  in 
coefficients  and  log  likelihood  values. 

3.  Lastly,  we  studied  relationships  of  chronic 
conditions  to  detailed,  aggregated,  and  global 
disability.  The  results  are  surprising  and 
welcome:  Chronic  conditions  are  excellent 
predictors  of  global  disability,  more  so  than  for 
aggregated  disability  (TABLE  3)  and  much  more  so 
than  for  detailed  items  (TABLE  4).  This  extends 
and  replicates  the  AHEAD  results  (which  showed 
that  aggregated  items  were  better  than  detailed 
ones).  On  this  basis,  we  can  state  a  hierarchy  of 
health-relatedness  for  disability  variables,  with 
global  disability  ranking  best  of  all.  Stroke, 
heart  disease,  and  psychiatric  problem  are  the 
strongest  predictors  of  global  disability.  Stroke 
also  proved  consistently  strong  in  the  AHEAD 
results. 

We  arrive  at  three  conclusions:  (1)  First, 
physical  limitations  are  the  foundation  for 
disability  in  midlife.  ADL  difficulties  are  not 
very  common  at  these  ages,  but  even  so,  their 
presence  is  much  less  predictive  of  global 
disability  than  are  physical  limitations.  Stated 
another  way,  generic  functional  problems  are  more 
implicated  in  general  disability  status  than  any 
specific  disabilities  are.  This  might  strike  some 
readers  as  odd — the  global  item  is  more  closely 
related  to  its  precursors  than  its  components. 
Whether  this  result  holds  up  in  older  samples  and 
in  data  sets  with  larger  arrays  of  physical, 
mental,  and  social  functioning  items  remains  to 
be  determined.  (2)  Second,  there  is  sizable 
overlap  between  global  disability  and  global 
morbidity.  This  result  was  expected.  Our  analyses 
are  very  simple,  and  a  firmer  judgment  of  what 
"sizable"  really  means  would  come  from  models 
using  both  global  disability  and  global  morbidity 
as  predictors  (X)  of  concurrent  or  prospective 
outcomes  (Y).  Because  global  items  are  rare,  we 
found  no  example  of  such  analysis  in  the 
literature;  there  are  examples  with  multiple  or 
aggregated  disability  items  as  predictors.  (3) 
Third,  global  disability  is  far  more  health- 
related  than  are  activity  domains  (ADLs,  physical 
limitations)  or  detailed  activities.  This  is  a 
welcome  result;  its  strength  surprised  us. 

A  Genuine  Global  Disability  Item  (Project  3: 
BRFSS) 

A  global  item  about  disability  was  included 
in  the  1993  Behavioral  Risk  Factor  Surveillance 
System  Survey  (BRFSS):  "During  the  past  30  days, 
for  about  how  many  days  did  poor  physical  or 
mental  health  keep  you  from  doing  your  usual 
activities,  such  as  self-care,  work,  or 
recreation?"  (Answer  range  is  0-30.)  BRFSS  is  a 
telephone  survey  of  adults  (ages  18+)  in  all  50 
states.  Four  general  items  about  health  and 
disability  were  introduced  in  the  1993  Core  and 
have  been  continued  since  then  (Hennessy, 
Moriarty,  Zack,  Scherr,  et  al.,  1994;  Centers  for 


Disease  Control  and  Prevention,  1994).  The  1993 
survey  also  had  a  special  module  on  Activity 
Limitations,  with  detailed  questions  about 
limitations  in  job,  housework,  personal  care 
(ADLs),  routine  needs  (IADLs),  and  any  other 
activities.  In  1993,  102,464  persons  responded  to 
the  Core;  for  analyses  we  use  a  20%  sample  (cases 
deleted  if  missing  data  for  age  and  disability), 
n=20,029.  The  activity  limitation  module  sample 
is  12,843. 

In  contrast  to  HRS,  BRFSS  gives  us  a  genuine 
global  item  to  study.  The  majority  of  persons 
(81.3%)  report  no  disability  days;  there  is  no 
difference  by  gender,  and  the  percent  rises  only 
slightly  across  age  groups.  Among  persons  with 
disability  days,  the  distribution  is  very  skewed 
with  reports  clustered  at  just  a  few  days  (1-3 
days;  47%)  or  else  all  30  days  (15%).  This 
suggests  that  the  item  picks  up  both  short-term 
and  long-term  disability. 

Further  descriptive  statistics  and  tables 
with  analytic  results  are  available  on  request. 
One  table  is  included  with  this  article.  A  paper 
is  in  preparation  for  journal  submission. 

1.  How  are  global  disability  and  self-rated 
health  related?  The  bivariate  relationship  is 
moderately  strong  and  in  the  expected  direction 
(the  more  disability  days,  the  worse  self-rated 
health).  The  correlation  is  .35  when  we  use  0-30 
disability  days,  and  .55  for  1-30  disability 
days;  thus,  having  some  disability  days  is  more 
indicative  of  health  status  than  0  days  (which 
can  occur  for  both  well  and  ill  persons).  We 
computed  logistic  regressions  with  self-rated 
health  and  sociodemographic/behavioral  factors  as 
predictors  of  global  disability  (0  vs  1+  days). 
Global  health's  association  with  global 
disability  diminishes  greatly,  both  in  odds 
ratios  and  explained  variance  (TABLE  5). 

2.  Which  activity  limitations  (job,  housework, 
ADL,  IADL)  are  well-represented  in  global 
disability?  We  used  cross-tabulations  to 
evaluate  these  associations.  The  global  item  was 
split  0  vs  1+  days  (none  vs  any),  and  then  also 
0-9  vs  10+  days  (none/short-term  vs  long-term 
disability).  Relationships  prove  to  be  much 
stronger  with  the  second  variable,  as  we 
expected:  The  activity  limitations  refer  to  long- 
term  disability,  and  the  0-9  vs  10+  split  of 
disability  days  captures  that  far  better  than  the 
0  vs  1+  split.  Associations  between  global 
disability  and  job  limitations  (esp.  being  unable 
to  work  at  all  due  to  health),  housework 
limitations,  and  IADL  limitations  are  very  high. 
The  association  with  personal  care  is  strong  for 
ages  70+  (but  only  moderate  for  ages  18-69). 
Compared  to  these,  associations  with  limitation 
in  any-other-activity  are  weak. 

3.  Parallel  cross-tabulations  of  the  activity 
limitations  with  self-rated  health  were  produced. 
The  associations  are  quite  similar  to  those  above 
(with  global  disability) .  With  this  exception: 
Needing  help  for  personal  care  or  routine  needs 
is  more  strongly  related  to  disability  days  than 
to  self-rated  health.  Stated  simply,  dependent 
persons  have  many  disability  days. 

We  come  to  three  conclusions:  (1)  First, 
once  personal  characteristics  associated  with 
both  are  taken  into  account,  we  see  that  global 
disability  is  distinctive  from  self-rated  health. 
That  is  reassuring,  given  the  less  certain 
findings  from  HRS  on  the  matter.  (2)  Global 
disability  reflects  underlying  specific 
disabilities  very  well.  It  should,  and  we  are 
pleased  the  data  verify  that.  The  only  exception 
is  limitations  in  "any  other  activities";  this  is 
a  vague  question  for  which  Yes  probably  means 
many  idiosyncratic  things.  (3)  Both  global 
disability  and  global  health  reflect  activity 
limitations,  but  global  disability  is  definitely 
better  in  reflecting  ADL  and  IADL  dependency. 

Conclusions 

Integrating  the  analyses  above,  we  come  to 
these  conclusions: 

1.  Detailed  Disability.  Plenty  of  detailed  items 
are  appropriate  in  surveys  if  every  single  one  of 
them  has  a  scientific  or  public  policy  rationale. 
Each  one  will  be  analyzed  on  its  own  at  some 
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point  to  fulfill  those  initial  purposes.  But  for 
more  general  analyses  of  the  data,  aggregated 
variables  such  as  "any  ADLs"  or  "sum  of  ADLs" 
have  better  analytic  yield.  (In  our  AHEAD 
analyses,  this  specifically  meant  stronger 
health-relatedness. ) 

In  many  population  health  surveys,  there  is 
no  good  rationale  for  having  numerous  detailed 
items  on  disability.  Will  just  a  few  do,  and 
which  ones?  The  AHEAD  results  suggest  that  the 
many  specific  ADLs,  IADLs,  or  physical 
limitations  have  similar  relationships  to  chronic 
morbidity,  so  choosing  any  small  set  of  them  will 
suffice  to  represent  disability.  This  is 
acceptable  for  a  cross-sectional  survey  setting. 
In  a  longitudinal  setting,  prediction  ability  as 
well  as  health-relatedness  must  come  into  the 
winnowing  decision.  One  needs  to  know  from  prior 
studies  if  detailed  items  have  similar 
prospective  prediction,  or  not. 

2.  Global  Disability.  A  global  disability 
indicator  reflects  the  disablement  process  very 
well.  The  HRS  results  show  that  it  has  stronger 
relationships  to  causal  precursors  (chronic 
conditions  and  physical  limitations)  than 
detailed  disability  or  aggregated  disability 
variables  do.  That  is  a  "plus'  in  its  favor.  But 
global  disability  has  strong  overlap  with  global 
morbidity  (self-rated  health).  The  extent  of 
overlap  needs  more  explicit  study  by  comparing 
the  two  items'  strength  as  predictors  of 
concurrent  and  prospective  outcomes.  At  issue  is 
the  net  effect  of  global  disability,  controlling 
for  self-rated  health. 

The  HRS  indicator  is  oddly  constructed  and 
not  ideal;  the  component  items  were  not  designed 
with  their  pooling  into  a  global  item  in  mind. 
Nevertheless,  the  indicator  is  analytically 
sturdy,  showing  distinctive  and  systematic 
results  when  compared  with  detailed  disability, 
aggregated  disability,  and  self-rated  health.  We 
have  no  doubt  that  overtly  designed  global  items 
will  do  as  well — and  likely  better. 

This  proves  true  with  the  BRFSS  item,  which 
was  designed  to  be  a  global  indicator.  Results 
show  that  it  does  a  very  good  job  in  capturing 
specific  disabilities.  We  prefer  a  global  item 
aimed  explicitly  at  long-term  disability  (the 
BRFSS  item  is  worded  to  include  both  short-  and 
long-term)  and  expect  such  an  item  to  show  even 
better  results,  that  is,  still  closer  coverage  of 
detailed  limitations. 

Recommendations 

Some  recommendations  for  research  and 
questionnaire  design  that  spring  from  our  work 
are  offered: 

1.  Recommendations far. Research.  What  research 

can  be  done,  economically  and  soon,  to  further 
the  goal  of  parsimonious  questioning  about 
disability?  We  make  four  recommendations: 

(1)  Existing  data  sets  with  numerous 
detailed  items  can  be  analyzed,  studying  closely 
item  correlations  and  scaling  characteristics. 
The  motivation  for  the  work  is  not  just 
psychometric  analysis  but  practical  decision- 
making about  (a)  items  that  can  be  dropped  or  (b) 
efficient  questioning  strategies  in  an  ordered 
series  of  items. 

(2)  With  imagination,  global  indicators  can 
be  generated  from  existing  data  sets.  Many 
surveys  now  have  series  of  activity  limitation 
questions  which  can  be  pooled  into  a  single 
variable  (as  NCHS  routinely  does  for  the  National 
Health  Interview  Survey)  .  Or,  numerous  detailed 
items  can  be  pooled  into  an  "any  disability" 
variable.  The  analytic  merits  of  these  pooled 
variables  can  be  compared  with  detailed  items. 

(3)  Surveys  with  genuinely  global  items  are 
few  and  far  between,  but  the  search  should  be 
made  and  opportunities  exploited.  We  are 
currently  analyzing  data  from  the  CDC  Behavioral 
Risk  Factor  Surveillance  System  survey  (BRFSS), 
which  included  global  morbidity  and  disability 
items  for  the  first  time  in  1993.  We  also  note 
the  two  Health  and  Activity  Limitation  Surveys 
conducted  in  Canada  (HALS)  and  the  1994-95 
Disability  Supplement  for  the  National  Health 
Interview  Survey  in  the  U.S.  (NHIS-Disability) ; 
we  leave  their  analytic  potential  to  readers' 
scrutiny. 


(4)  Global  items  can  be  crafted  and  then 
evaluated  for  colloquial  sense  and  content  in 
laboratory  settings.  A  crucial  aspect  of  this 
work  is  to  determine  the  best  place  for  the  two 
essential  qualifiers  (disability  is  protracted 
and  health-related).6  Nonverbal  formats  such  as 
the  COOP  chart  of  Daily  Activities  (Beaufait, 
Nelson,  Landgraf,  Hays,  Kirk,  Wasson,  et  al., 
1992)  should  be  considered  and  tested  in 
conjunction  with  verbal  ones. 

2.  Recommendations  for  Survey  Design.  What  can  be 
done  immediately,  without  additional  research 
information,  when  designing  surveys? 

(1)  Every  survey  that  includes  self-rated 
health  should  also  include  a  global  disability 
item.  The  briefest  rationale  is  that  "functional 
status  is  just  as  important  as  health  status". 
Items  used  in  other  surveys  to  date  are  shown  in 
Verbrugge  (1994),  and  good  candidates  are  noted 
for  consideration  in  future  surveys.  They  are 
appended  here  as  Figure  1. 

(2)  If  detailed  items  are  needed,  every  one 
should  have  excellent  rationale  and  conceptual 
integrity.  Its  analytic  use  should  be  known  in 
advance  (if  it  doesn't  exist,  neither  should  the 
item) .  The  conceptual  niche  that  each  holds 
should  be  stated  clearly.  Further,  overall 
coverage  of  the  concept  "disability"  should  be 
considered  afresh  when  a  survey  is  designed.  This 
means  resisting  the  pressures,  which  are  very 
strong,  to  repeat  items  used  in  other  surveys. 
For  example,  if  n  questions  are  desired,  surveys 
can  have  better  coverage  of  the  disability 
experience  by  asking  about  more  activity  domains 
and  just  one  dimension,  in  contrast  to 
contemporary  practices  of  asking  about  few 
domains  and  several  dimensions. 

Summing  up,  the  goal  is  to  measure 
disability  in  comprehensible,  comprehensive, 
veridical,  and  useful  ways  in  health  surveys.  We 
think  it  can  be  done  with  more  parsimony  than  now 
exists. 
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Table  1.  Effects  of  Specific  Disability  Items  on 
Global  Indicator,  Logistic  Regression:  American 
Adults 


Explanatory 

Model 

Model 

Model 

variables 

I 

II 

III 

ADL  items 

Walk 

0.685 

2.343 

Bathe 

0.944 

2.281 

Transfer 

0.697 

1.807 

Eat 

-0.918 

0.384 

Dress 

0.794 

1.488 

Physical  limitation 

items 

1.045 

Walk  several  blocks 

0.964 

Climb  several  stairs 

0.704 

0.869 

Pull/push  objects 

1.214 

1.277 

Lift/carry  10  pound 

0.945 

1.024 

Pick  up  a  dime 

0.398 

0.569 

Intercept 

-2.829 

-2.353 

-2.762 

Log  Likelihood 

4058.03 

2063.80 

3985.42 

Sample  size 

11972 

12429 

12035 

Table   2.   Association   between   Global   Health 
Indicator  and  Global  Disability  Indicator 


Explanatory 
variables 


Model 

I 


Model 
II 


Model 
III 


Poor  self-rated  health 


ADL  indicator 
Physical 

limitation 
Global  disability 
Intercept 
Log  likelihood 


2.626 


-1.936 
1646.52 


2.911 

3.455 
-2.623    -2.589 
1434.72   2250.01 


Poor/fair  self-rated  health 

ADL  indicator        2.318 

Physical  limitation  2.026 

Global  disability  2.418 

Intercept         -0.791  -0.861  -0.707 

Log  likelihood     2350.69  2542.08  3565.42 

Sample  size         12443  12053  12598 


Table  3 .  Effects  of  Specific  Chronic  Conditions 
on  Three  Disability  Indicators 


Stroke 

0.673 

0.836 

0.928 

Psychiatric 

0.341 

0.485 

0.444 

problem 

Arthritis 

0.505 

0.322 

0.236 

Kidney  disease 

0.308 

0.201 

0.349 

Other  diseases 

0.476 

0.495 

0.456 

Intercept 

-2.849 

-3.000 

-3.724 

Log  likelihood 

2752.55 

2863.12 

2861.37 

Sample  size 

12114 

12093 

12119 

Table  5.  Logistic  Regression  of  Self-Rated  Health 
and  Risk  Factors  on  Disability  Days  (0  vs  1-30) 


Model 

Model 

Model 

I 

II 

III 

Beta 

OR 

Beta 

OR 

Beta 

OR 

Self-rated 

Health: 

Very  Good 

.57 

1. 

8 

.22 

1.3 

Good 

.77 

2. 

2 

.47 

1.6 

Fair 

1 

.53 

4. 

6 

.82 

2.3 

Poor: 

2 

.74 

15. 

,5 

1.59 

4.9 

Age  (#  yrs) 

-.003 

1.0 

-.01 

1.0 

Gender 

-.04 

1.0 

-.04 

1.0 

Education 

-.001 

1.0 

.07 

1.1 

Working 

-.28 

.8 

-.20 

.8 

Retired 

-.31 

.7 

-.30 

.7 

Income 

-.09 

.9 

-.03 

1.0 

Insurance 

.16 

1.2 

.18 

1.2 

Married 

.08 

1.1 

.09 

1.1 

White 

-.19 

.8 

-.13 

.9 

Black 

.08 

1.1 

.10 

1.1 

Hispanic 

-.04 

1.0 

-.03 

1.0 

Body  Mass 

Index 

-.0001 

1.0 

-.001 

1.0 

Diabetes 

.39 

1.5 

.21 

1.2 

High  Blood 

Pressure 

.22 

1.3 

.13 

1.1 

Alcohol  Use 

(any) 

-.02 

1.0 

.02 

1.0 

Mental  Health 

Disab  Days 

.92 

2.5 

.86 

2.4 

Physical  Health 

Disab  Days 

2.56 

1.9 

2.43 

11.4 

Current  Smoker 

.06 

1.1 

-.01 

1.0 

Former  Smoker 

.13 

1.1 

.10 

1.1 

Log-likelihood  221305383 
R2  .07 


861032406 
.28 


892200790 
.29 


Explanatory    ADL      Physical    Global 
variables    indicator   limitation   indicator 


Hypertension 

0.113 

0.310 

0.211 

Diabetes 

0.681 

0.644 

0.568 

Cancer 

0.386 

0.360 

0.652 

Lung  disease 

0.301 

0.756 

0.731 

Heart  disease 

0.269 

0.725 

1.012 

Stroke 

1.102 

0.878 

1.533 

Psychiatric 

0.423 

0.463 

0.821 

problem 

Arthritis 

0.687 

0.380 

0.586 

Kidney  disease 

0.394 

0.284 

0.492 

Other  diseases 

0.561 

0.483 

0.625 

Intercept 

-3.604 

-2.376 

-2.919 

Log  likelihood 

2141.77 

3176.14 

4119.18 

Sample  size 

12091 

11711 

12245 

Table  4.  Effects  of  Specific  Chronic  Condition  on 
Three  Disability  Items 


Figure  1. 
indicator 


Candidates  for  a  global  disability 


1.  National  Population  Health  Survey-  Canada 

'The  next  few  questions  deal  with  any  health 
limitations  which  affect  ...'s  daily  activities. 
In  these  questions,  'long-term  conditions'  refer 
to  conditions  that  have  lasted  or  are  expected  to 
last  6  months  or  more'. 

'Because  of  a  long-term  physical  or  mental 
condition  or  a  health  problem,  are  you  limited  in 
the  kind  or  amount  of  activity  you  can  do: 

At  home  ? 

At  school? 

At  work? 

In  other  activities  such  as  local  travel,  sports 

or  leisure?' 


Explanatory 
variables 


Climb 
stairs 


Pull/push 
objects 


Lift/carry 
10  pounds 


Hypertension 

0. 

,367 

0, 

.151 

0, 

,257 

Diabetes 

0. 

,593 

0, 

.451 

0. 

.434 

Cancer 

0. 

,264 

0, 

.418 

0. 

.362 

Lung  disease 

0. 

,829 

0. 

.504 

0. 

,539 

Heart  disease 

0. 

,632 

0. 

.740 

0. 

.669 

For  each:  yes,  no 

Note:  Canada  has  used  global  disability  items  in 
its  population  census  and  in  national  surveys 
such  as  the  Health  and  Activity  Limitation 
Surveys  1986-87  and  1991  and  the  National 
Population  Health  Survey  1994-95.  The  items  have 
been  very  similar,  with  a  little  modification 
from  one  census/survey  to  the  next.  We  show  the 
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contemporary  item  but  alter  the  descriptor  for 
other  activities  from  'such  as  transportation  to 
or  from  work  or  leisure  time  activities'  to  'such 
as  local  travel,  sports  or  leisure',  a  close 
reprise  of  what  appears  in  some  of  the  prior 
surveys. 

2.  New  (developed  by  author) 

'Because  of  a  physical,  mental,  or  emotional 
condition,  are  you  limited  in  doing  your  daily 
activities  like  personal  hygiene,  house  or  yard 
care,  shopping,  your  work,  or  other  things  you 
need  to  do?'  Yes,  no 

If  yes:  'Has  the  limitation  lasted  for  at  least 
6  months  or  is  it  expected  to  last  that  long? ' 
Yes,  no.  If  yes  to  6+  months:  'Are  you  limited 
just  a  little,  somewhat,  or  a  great  deal  in  your 
daily  activities?'  Just  a  little,  somewhat,  a 
great  deal. 

Note:  The  item  covers  many  domains,  has  a  six 
month  reference  period  for  disability,  and  has 
severity  gradations. 

3.  Modified from  a  pilot Study  on subjective 

health 

•Is  there  anything  about  your  health  that  makes 
it  hard  for  you  to  do  your  usual  activities?' 
Yes,  no. 

If  yes:  'Has  the  difficulty  with  your  activities 
lasted  6  months  or  more,  or  do  you  expect  it  to 
last  that  long?'  Yes,  no. 

If  yes  to  6+  months:  'What  are  the  activities  you 
have  trouble  doing  because  of  health? ' 
Interviewer  records  responses. 

'Would  you  say  your  difficulty  doing  these 
activities  is  a  little,  some,  or  a  lot?'  A 
little,  some,  a  lot. 

Note:  The  item  is  modified  from  a  small-scale 
pilot  study  conducted  by  Charles  Cannell  and 
colleagues  for  the  National  Center  for  Health 
Statistics  in  1975.  We  simplify  the  lead 
question,  add  the  reference  period  for 
disability,  and  use  different  severity 
gradations. 

4.  Modified  from  National  Health  Interview  Survey 
Disability  Supplement.  United  States.  1994-95 


After    specific    questions 
conditions,  if  yes  to  any: 


about    physical 


■During  the  past  12  months,  did  any  of  these 
problems  seriously  interfere  with  your  ability 
to  work  or  attend  school  or  to  manage  your  day- 
to-day  activities?'  Yes,  no. 

After  specific  questions  about  cognitive  and 
emotional  problems,  if  yes  to  any:  (same 
question) 

Note:  The  supplement  accompanies  the  1994-95 
National  Health  Interview  Survey  (NHIS).  It  has 
two  phases:  phase  one  occurs  at  the  same  time  as 
the  NHIS  Core  and  has  disability  questions  about 
all  household  members;  phase  two  is  conducted 
several  months  later  for  persons  who  screen  in 
from  phase  one  as  having  disabilities.  Here,  we 
use  a  phase  one  item  about  emotional/cognitive 
problems,  adding  a  parallel  one  about  physical 
problems. 

5.  Modified  from  the  Baltimore  Longitudinal  Study 
of  Aging  Followup  1 

'Would  you  describe  your  overall  level  of 
functioning  in  your  home,  work,  and  leisure 
activities  as:  excellent,  very  good,  good,  fair, 
poor,  don't  know?'  Excellent,  very  good,  good, 
fair,  poor. 

Note:  The  Baltimore  Longitudinal  Study  is  a  life- 
long study  of  adults  conducted  by  the  Gerontology 


Research  Center,  National  Institute  on  Aging. 
Participants  have  medical  exams  and 
questionnaires  every  two  years.  The  Follow-up  was 
conducted  in  1989  on  dropouts,  people  who  had  not 
returned  for  the  biennial  exam.  We  modify  the 
item  by  adding  the  descriptor  'in  you  home,  work 
and  leisure  activities'  and  including  the 
category  'very  good'  (to  match  the  five  response 
categories-excellent,  very  good,  good,  fair, 
poor-now  used  for  self-rated  health  items  in  the 
US).  This  question  must  be  asked  in  the  context 
of  health/functioning;  without  that  context, 
"functioning"  is  vague. 


Lois  M.  Verbrugge  is  a  Distinguished  Research 
Scientist,  Susan  S.  Merrill  is  a  Postdoctoral 
Fellow,  and  Xian  Liu  is  an  Assistant  Research 
Scientist  at  the  Institute  of  Gerontology, 
University  of  Michigan,  Ann  Arbor,  MI.  This 
article  extends  the  analyses  and  text  of  a  prior 
presentation,  published  in  the  Proceedings  of  the 
Sixth  Conference  on  Health  Survey  Research 
Methods  (Breckenridge,  CO),  June,  1995. 

Footnotes 

:We  distinguish  them  from  two  other  formats:  (1) 
An  aggregated  item  adds  up  the  number  of  specific 
disabilities.  This  is  analytically  compact,  but 
not  compact  in  the  questionnaire  itself.  (2)  A 
short-form  instrument  covers  multiple  diverse 
concepts  about  health  and  functioning  with  about 
5-20  questions  total.  By  contrast,  a  global 
indicator  covers  just  one  concept. 

Conceptually,  physical  limitations  are  aspects  of 
functional  limitation,  not  disability  (Verbrugge 
&  Jette,  1994).  For  sake  of  economy,  this  is  not 
emphasized  in  the  paper. 

Questionnaire  items  for  determining  presence/ 
absence  vary  for  the  conditions;  for  example, 
physician  diagnosis  of  condition,  own  statement 
about  presence  of  condition,  symptoms  in  past 
year.  Details  can  be  found  in  AHEAD  documents  (it 
is  a  public-use  data  set)  or  the  manuscript 
cited. 

4The  HRS  analyses  were  conducted  by  authors 
Verbrugge  and  Liu. 

5Because  of  the  sequential  questioning,  the  two 
items  on  housework  limitations  and  other-activity 
limitations  cannot  be  analyzed  on  a  whole-sample 
basis,  and  prevalence  rates  for  them  cannot  be 
estimated  from  HRS. 

6The  several  options  are  to  place  these  qualifiers 
in  an  initial  preface  (asking  respondents  to 
think  about  long-term  health-related  problems  in 
the  following  questions),  in  each  question,  or  in 
Follow-up  probes  (checking  about  duration  and 
health-relatedness  after  respondents  say  yes  to 
the  disability  question).  Which  approach  achieves 
and  maintains  the  desired  focus  without  excessive 
verbiage? 
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INTRODUCTION 

Despite  significant  echnological 
and  medical  advances  in  diagnosis  and 
treatment,  racial  and  ethnic 
disparities  in  health  status  and 
access  to  health  care  persistent  as 
public  health  concerns.   Differences 
in  guality  of  health  care  received 
have  been  well  documented,  with  the 
low  income  and  black  patients 
receiving  worse  care. 
To  properly  identify  and  interpret 
the  causes  of  these  disparities, 
improvements  in  the  reliability  and 
cultural  appropriateness  of  existing 
data  collection  instruments  and 
assessment  of  the  data  collected  must 
occur. 

Concerted  efforts  have  been  made 
to  over  sample  various  minority 
population  groups.   Nevertheless,  the 
needs  of  minorities  and  special 
populations  remain  inadeguately 
assessed  and  integrated  into  data 
collection  efforts.   The  theme  of  the 
1995  Public  Health  Conference  on 
Records  and  Statistics  and  the 
National  Committee  on  Vital  and 
Health  Statistics  45th  Anniversary 
Symposium,  convened  by  the  National 
Center  for  Health  Statistics  (NCHS) 
was  "Data  Needs  in  an  Era  of  Health 
Reform."  The  purpose  of  this  paper  is 
to  discuss  approaches  and 
methodologies  for  improving  the 
collection  and  analyses  of  data  on 
the  health  of  minorities  and  other 
special  populations. 

BACKGROUNG 

The  Report   of  the  Secretary's 
Task  Force  on  Black  and  Minority 
Health.      released  in  1985  is  the  most 
significant  official  Federal 
publication  which  documents 
disparities  in  health  status  and 
access  to  adeguate  health  care  among 
racial  and  ethnic  populations  in  the 
United  States.   The  Report  served  as 
the  official  impetus  for  research  and 


public  health  policies  to  document, 
understand,  and  improve  minority 
health  in  this  Nation.   Many  of  the 
determinants  of  the  observed 
disparities  identified  then  (e.g., 
SES,  poverty,  race,  ethnicity, 
barriers  to  appropriate  care,  etc.) 
still  remain  national  problems. 

Efforts  to  systematically  reform 
health  care  and  public  health 
nationwide,  however,  are  not  new.   In 
1974,  President  General  Ford  signed 
into  law  the  National  Health  Planning 
and  Development  Act  (P.L.  93-641) , 
authorizing  a  three-year  health 
planning  and  facilities  program. 
Title  XV  of  the  Act  established 
national,  state,  and  local 
organizations  for  health  planning  and 
combined  existing  health  planning 
programs.   The  second  part  of  the 
Act,  Title  XVI,  revised  and  extended 
the  medical  facilities  construction 
(Hill-Burton)  program  and  authorized 
funds  for  developing  health  resources 
nationwide.   The  1994  Act  established 
a  network  of  regional  Health  Systems 
Agencies  (HSAs)  for  areawide  health 
planning  and  development  in  health 
services  areas  to  be  designated  by 
State  Governors. 

In  1990,  after  three  years  of 
development  and  several  public 
hearings,  then  Secretary  of  Health, 
Dr.  Louis  Sullivan,  implemented  a  new 
national  strategy  for  significantly 
improving  the  health  of  all  Americans 
—  with  particular  focus  on 
prevention.   The  document,  entitled 
Healthy  People  2000:    National   Health 
Promotion  and  Disease  Prevention 
Objectives,    contained  over  400 
health-related  objectives   grouped 
into  22  priority  areas  to  be  achieved 
by  or  before  the  Year  2000. 
Ambitious  as  many  of  the  objectives 
are,  approximately  100  new 
subobjectives  and  data  sources  for 
minority  populations  were  added  in 
1994  in  response  to  a  midcourse 
reviews  of  the  original  objectives. 
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Also  in  1990,  the  U.S.  Congress 
passed  the  Disadvantaged  Minority 
Health   Improvement  Act   of   1990.    The 
Act  provides  national  guidelines  and 
laws  for  health  services  programs, 
research  and  data  collection  and 
analysis  aimed  to  reduce  and 
alleviate  the  disproportionate  burden 
of  illness,  disabilities,  and  deaths 
borne  by  racial  and  ethnic  minority 
populations  int  the  United  States. 
The  Act  aimed  also  to  institute 
Federal,  State,  and  community-based 
health  programs  for  removing 
socioeconomic,  linguistic,  and 
cultural  barriers  to  appropriate 
health  care  for  member  of  the  various 
minority  population  groups.   The 
Americans  with  Disability  Act  of  1990 
seeks  also  to  do  the  same  for  the 
over  20  million  Americans  with 
various  types  and  degrees  of 
disabling  (health)  conditions. 

The  current  debate  in  this 
health  care  reform  era  is,  largely 
about  approaches  for  ensuring  access 
to  adequate  health  care,  at  a 
reasonable  cost,  for  everyone  in  the 
US  without  compromising  quality.   The 
concern  is  especially  for  low-income 
and  medically  indigent  persons  who 
are  traditionally  underserved  or  left 
out  of  the  existing  predominant  fee- 
for-service  health  care  system.   The 
focus  of  this  1995  NCHS-  sponsored 
conference  appropriately  must  address 
records  and  statistics  needed  for 
health  reform  relative  to  emerging 
public  health  issues.   To  be  most 
relevant  to  the  current  National 
debate  the  conference  must  highlight 
the  critical  and  growing  role  of  data 

in  the  changing  health  care  system. 

ASSESSING  THE  HEALTH  OF  SPECIAL 
POPULATIONS 

Health  needs  assessment  requires 
succinct  definition  of  a  desired 
state  (target)  of  "health"  and  an 
accurate  knowledge  of  the  present 
(baseline  health  profile  of  the 
community  or  individual.   Data  from 
existing  national  data  collection 
systems  maintained  by  NCHS  and  other 
federal  agencies  were  used  in 
establishing  the  baseline,  and  indeed 
most  of  the  Year  2000  health  status 
targets  for  the  various  subpopulation 
groups.   Some  of  the  national 
population-based  and  vital  statistics 
data  collection  systems  can  be  used 
to  get  a  general  idea  of  the  health 
status  of  Americans.    However,  these 
data  are  neither  adequate  nor 
appropriate  for  assessing  the  health 
needs  of  minorities  and  other  special 
populations  at  State  and  local 
(community)  levels. 


The  three  major  goals  of 
"Healthy  People  2000"  are  to  (1) 
increase  the  span  of  healthy  life  for 
Americans;  (2)  reduce  health 
disparities  among  Americans;  and,  (3) 
achieve  access  to  preventive  services 
for  all  Americans.   A  special 
category  of  seven  objectives  were 
established  to  improve  public  health 
surveillance  and  data  systems  by  the 
year  2000.   Compared  to  their  white 
counterparts,  blacks  in  America  have 
always  had  shorter  life  expectancy 
which  has  been  consistently  5  to  8 
years  shorter  than  whites  in  the  last 
two  decades,  1970-1990.   The  life 
expectancy  at  birth  for  blacks  in 
1990  (  69.1  years  of  life)  is  7  years 
less  than  that  of  whites  (76.1 
years) ,  and  approximates  where  white 

Americans  were  forty  years  ago 

life  expectancy  of  whites  birth  was 
69.1  in  1950.   In  1990  Black  females 
could  expect  to   live  approximately  9 
years  longer  than  black  males  born 
the  same  year;  but  6  years  shorter 
than  their  white  counterparts  on  the 
average.   (Health,  USA,  1993) .   For 
American  aged  74  years  and  younger  in 
1993,  the  age-adjusted  death  rate  for 
blacks  was  615  per  100,000  population 
compared  with  352  per  100,000 
population  for  white  Americans. 

Twice  as  many  black  babies  do 
not  live  to  their  first  birthdays, 
compared  to  their  white  counterparts. 
The  infant  mortality  rates  for  blacks 
have  been  at  least  twice  that  of 
whites  during  1970  through  1990;  32.6 
and  18.0  deaths  per  1,000  live  births 
for  blacks  compared  with  17.8  and  7.6 
deaths  per  1,000  live  births  for 
whites  in  1970  and  1990, 
respectively.   (NCHS,  Health  USA, 
1994) .   In  terms  of  access  to  and 
utilization  of  appropriate  health 
care  services,  fewer  pregnant  black 
women  receive  or  initiate  prenatal 
care  during  the  first  trimester  of 
their  pregnancies  compared  with  their 
white  counterparts  (NCHS, Health  USA, 
1994) .   Compared  to  whites,  a  higher 
percentage  of  blacks,  Hispanics,  and 
low-income  Americans  lack  a  regular 
source  of  primary  health  care. 
(NCHS,  NHIS,  1992) . 

Mid-course  reviews  conducted  by 
the  NCHS  and  the  US  Public  Health 
Services  to  review  progress  toward 
achieving  the  Year  2000  objectives 
(NCHS,  1994)  and  statistical 
projections  based  on  1990-1993  data 
predict  that  all  minority  population 
groups  will  not  only  miss  several  of 
their  Year  2000  objectives,  but  will 
actually  be  worse  off  in  year  2000 
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Figure  1.  Life  expectancy  at  birth  by  race  and  sex:  United  States,  1970-91 
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Race  and  sex 


1970 


1975 


1980 


1985 


1988 


1989 


1990 


1991 


White  male 68.0  69.5  70.7  71.8  72.2  72.5  72.7 

Black  male 60.0  62.4  63.8  65.0  64.4  64.3  64.5 

White  female 75.6  77.3  78.1  78.7  78.9  79.2  79.4 

Black  female 68.3  71.3  72.5  73.4  73.2  73.3  73.6 

SOURCE:  Centers  for  Disease  Control  and  Prevention,  National  Center  for  Health  Statistics,  National  Vital  Statistics 
System.  See  related  Health,  United  States,  1993,  table  27. 
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than  their  health  profiles  indicate 
they  are  in  1995.   For  example, 
objective  14.1  of  "Healthy  People 
2000"   is  to  reduce  infant  mortality 
rate  to  no  more  than  7  per  1000  live 
births  for  all  Americans  by  Year  2000 
(from  a  baseline  of  10.1  per  1000 
live  birth  in  1997), and  to  reduce  the 
black  infant  mortality  rate  to  no 
more  than  11  per  1000  live  births 
(from  a  baseline  of  17.9  per  1000 
live  births  in  1987) . 

Regression  analysis  of  1970-81 
and  1981-88  data,  however,  project 
within  95  percent  confidence  interval 
that  both  targets  for  infant 
mortality  rates  will  be  missed,  given 
current  statistical  pattern.   The  gap 
in  infant  mortality  rates  between 
white  and  black  Americans  will 
actually  widen  by  Year  2000  if  the 
current  trend  continues.   (Feinleib, 
1993) . 

Assessing  the  health  needs  of 
minorities  and  other  special 
population  is  methodologically 
complex  and  difficult.   But,  it  must 
be  done  well.   Policy  and  program 


discussion  must  be  based  on  accurate 
and  reliable  data  if  appropriate 
health  programs  are  to  be  instituted 
to  reduce  the  disparities  in  health 
status  among  Americans.   The 
challenge  for  health  planners  and 
public  health  practitioners  in 
general  is  the  fact  that  Year  2000  is 
only  five  years  away  and  much  remains 
to  be  done. 

Much  of  the  debate  about  health 
care  reform  concern  medical  care 
services,  albeit  at  the  expense  of 
public  health  services.   A  national 
objective  (#22.4)  for  Year  2000, 
however,  is  the  development  and 
implementation  of  a  national  process 
to  identify  significant  gaps  in  the 
national  disease  prevention  and 
health  promotion  data,  including  data 
for  racial  and  ethnic  minorities, 
people  with  low  income,  and  people 
with  disabilities.   Mechanisms  must 
then  be  established  to  meet  these 
needs.   (HP  2000  objectives,  U.S.  - 
PHS) .   Disease  prevention  and  health 
promotion  data  in  this  context 
include  data  on  various  disease 
status,  risk  factors,  and  health  care 
services  received.   Public  health 
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problems  include  issue  areas  such  as 
HIV  infection,  domestic  violence, 
mental  health,  environmental  health, 
occupational  health,  and  disabling 
conditions. 

Comprehensive  assessment  of  the 
health  needs  of  minorities  and  other 
special  populations  require  that 
specific  standardized  data  elements 
be  routinely  collected  in  all  the 
areas  previously  listed.   The  data 
elements  must  be  integrated  into 
standardized  data  sets.   The  office 
of  the  Associate  Director  for 
Minority  Health,  Centers  for  Disease 
Control  and  Prevention  (CDC) ,  in 
collaboration  with  the  PHS-Office  of 
Minority  Health  (PHS-OMH) ,  the  NCHS, 
and  the  Regional  Minority  Health 
Research  Center  at  Morehouse  School 
of  Medicine  are  engaged  in  an  on- 
going project  known  as  the  "A 
Critical   Review  of  Status  and  Trends 
in   the  Health   and  Quality  of  Life  of 
Racial   and  Ethnic  Minority  Population 
in   the   U.S.".      Initiated  in  1991,  the 
project  seeks  to  update  the  status  of 
minority  health  in,  the  U.S.  since 
that  release  of  the  1985  Report  of 
the  Secretary 's  Task  Force  on  the 
Health  of  Blacks  and  other  minority 
populations.   The  project  uses,  and 
will  enhance,  existing  surveillance 
systems  to  monitor  progress  toward 
the  Year  2  000  objectives  and  beyond 
for  racial  and  ethnic  minority 
populations.   An  aim  of  the  "Critical 
Review  Project"  is  to  develop  a 
systematic  method  for  assessing  the 
adequacy  of  existing  data  systems  for 
monitoring  minority  health  status  and 
quality  of  life  indicators.   In  1994, 
an  update  of  excess  deaths  among 
blacks  and  Hispanic  was  published  by 
NCHS.   Under  the  supervision  of  the 
Senior  Epidemiologist  for  Minority 
Health,  research  assistants  at  the 
Morehouse  School  of  Medicine's 
Regional  Minority  Health  Research 
Center  developed  a  stepwise  approach 
in  Component  II  of  this  initiative  to 
assess  the  adequacy  of  existing  data 
systems  for  minority  morbidity  and 
disability  among  the  various  racial 
and  ethnic  minority  population  group. 
The  steps  include: 

Step  1.   Complied  Diseases  and  Health 
condition  -  specific  measures  and 
Indicators  (HP  2  000  objective  and 
Critical  review  Project; 
Step  2.   Compiled  Statistical 
Variable/Data  Elements  needed  to 
compute  the  Measures/Health 
Indicators  (above) ; 

Step  3.   Identified  Sources  of  Data 
(Data  Systems)  containing  the  data 
elements,  (published  and  electronic 
sources) ; 


Step  4.   Developed  Criteria  for 

assessing  the  adequacy  of  existing 

data  systems  for  computing  specific 

measures  of  health  status; 

Step  5.   Reviewed  documentation/ 

detailed  descriptions  for  various 

data  systems ; 

Step  6.   Identified  Gaps/ 

Inadequacies  in  existing  data  systems; 

Step  7.   Developed  Recommendations 

for  filling  the  gags. 

Disease-and  health  condition- 
specific  measures  and  indicators  were 
compiled  for  HIV/AIDS,  the  six 
diseases  and  health  conditions 
highlighted  in  the  1985  report 
(cardiovascular  diseases  and  stroke, 
cancer,  homicide,  infant  mortality) , 
and  the  cross-cutting  issues  (access 
to  and  financing  of  health  care,  and 
health  professions) . 

SOURCES  OF  DATA  FOR  SELECTED  HEALTH 
STATUS  INDICATORS 


STATISTICAL 

MEASURES 

STATISTICAL 
VARIABLES 

PROPOSED  DATA 
SOURCES 

Survival  rates  for  all 
major  cancers  affecting 
minorities 

SES 

Smoker/  nonsmoker 

Sex 

Age 

Demographic  area 

Ethnicity 

SEER 
NAMCS 
NHIS 
BRFSS 

Prevalence  of  tobacco 
use 

SES 
Sex 
Age 

Demographic  area 
Incidence  of  lung  cancer 
Elhnicilv 

SEER 

NHIS 

NHANES 

BRFSS 

HHANES 

YRBS 

Prevalence  of  IV  drug 
use 

SES 
sex 

ethnicity 
demograhic  area 
drug  abuse  and  type 

NHSDA 

DAWN 

MFS 

HIV  Sero  in  Drug  Treatment 

Centers 

PrvalcnceofHTV 
infection 

SES 

ethnicity 
demograhic  area 

NHIS 

NHDS 

NHANES 

AIDS  Surveillance 

HIV  Counseling  and  Testing 

Prevalence  of  condom 
use  in  the  sexually  active 

SES 

ethnicity 
demograhic  area 
knowledge  /attitude  about  AIDS 
sexual  practices/attitude 

NHIS 
NSFO 

Minorities  reporting 
environmental  or 
chemical  exposures  at 
work  or  home 

SES 
Sex 

Age 

Demographic  area 
Smoker/nonsmoker 
Incidence  of  lung  cancer 
Type  of  exposures 
Ethnicity 

SEER 

NHIS 

NHANES 

NOES 

HHANES 

NHDS 

NOHS 

OED 

Long  term  disability 
(mental  or  physical) 

SES 
Sex 

Ethnicity 
Demographic  area 

NHIS 

NHANESI 

NNHS 

NHHCS 

LSOA 

Impact  on  Family  and 
Significant  Others 

SES 
Sex 

Ethnicity 

Demographic  area 

Stress 

Depression 

Days  lost  from  work/school 

NHANES 
NHIS 

Implications  for 
Employment 

SES 
Sex 

Ethnicity 
Demographic  area 
Depression . 
Loss  of  Job 

NHANES 
NHIS 
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More  importantly,  a  set  of 
criteria  was  developed  for  assessing 
the  adequacy/  utility  of  existing 
data  systems  for  use  in  computing  the 
specific  health  status  indicators. 
Adequacy  of  a  data  system  for 
minority  health  depends  mainly  on  the 
variables  and  data  elements  in  the 
data  system,  as  well  as  the  size  and 
ethnic/national  (origin)  diversity  of 
minority  populations  included. 

Selected  criteria  for  assessing 
adequacy  of  existing  data  systems  for 
minority  health 

1.  Whether  the  data  system  contains 
variables  (data  elements)  for  health 
measures  of  interest; 

2.  Whether  incidence  and  prevalence 
rates  are  attainable; 

3.  Appropriateness  of  sampling 
frame, 

4.  At  the  minimum,  contains  racial 
and  ethnic  classifications  as  defined 
by  OMB  Directive  15; 

5.  Number  of  minorities  in  survey 
samples,  whether  oversampled  minority 
population  groups; 

6.  Periodicity  of  data 
collection;  and, 

7.  Availability  of  comparable 
denominator. 

Information  needed  for 
comprehensive  assessment  of  the 
health  needs  of  a  community  should 
include: 

1.  Geographic  (Regional) 
Information 

PHS  Region 

State 

County 

SMSA 

City  -  Zip  Code 

-  Census  Tract 

-  Street  Name 

-  Household  Address 

2.   Demographic  Information 

Age,  Gender 
Race 

Ethnicity/National  Origin 
(i.e., Cultural  Background) 
Native  born/Foreign  born 
Generation  represented  in  the  U.S. 
Duration  of  Residency 
-in  the  geographic  region 
-in  the  U.S.  (if  "foreign 
born") 

Marital  Status 
Socio-economics 
-year  of  schooling  completed 
-type(s)  of  employment,  etc.  (if 
employed) 

-monthly  and  annual  income 
(net  income) 


Detailed  Household 
Information 

-number  of  persons  in  the 
household 

-head-of -household 
-name,  age,  gender  of  persons  in  the 

household 
-literacy  in  native  language 
-literacy  in  English 

3.  Health  Status  Indicators 
(partial  List) 

-Prevalence  of  Coronary  Artery 

Disease 
-Numbers  of  ER  Visits  for  Angina  and 

Chest  Pain 
-Prevalence  of 

Hypercholesterolemia 

(>or=240  mg/dl) 
-Prevalence  of  Obesity 
(m>or=27.8;  f>or=27.3) 
-Prevalence  of  Diabetes 

Mellitus 
-Proportion  of  Regular, 

Moderate,  or  Light  Exercise 
-Number  of  children  <2  years 

immunized 
-Impact  of  Family  and 

Significant  Others 
-Implications  for  Employment 
-Prevalence  of  Hypertension 
-Health  care  expenditure  for 

rehabilitation  services  for  post- 
stroke  victims 
-Long  term  disability  (mental  or 

physical) 

4.  Community  Resources/Structure 

A.  Administrative 
-local  political 

structure 

B.  Programmatic 
Resources  facilities  and 
Personnel) 

-Social  Services,  Personnel  and 

Facilities 
-Health  Services, Personnel,  and 

Facilities 

(Types  of  services  available, 

number  of  beds, 

proprietorship,  number  &  types  of 

personnel,  #MDs/other  providers 

accepting  medical/medicare, 

etc. , ) 

-Educational  Services, 

Personnel  and  Facilities 

-Public  Services/Safety, 

Personnel  and  Facilities 

5.  Environmental  Information 

(at  Census  Tract/Zip  Code  level) 
-Types  of  Business; 
-Size  of  Company  (No. ,  race,  age, 

gender,  and  national  origin  of 

employees; ) 
-Industries  (types,  types  of 

products,  types/  levels  of  emissions^ 
-Types  of  pollutants  and  pollution 

levels  (daily/monthly  average) 
-Water  Supply  System  (Sources) 
-Landfills  (location  and  type) . 
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LIMITATIONS  OF  EXISTING  DATA  SYSTEMS 
FOR  ASSESSING  MINORITY  HEALTH  NEEDS 

Existing  (national)  population- 
based  data  systems   are  neither 
adequate  nor  appropriate  for 
assessing  health  needs  of  minority 
and  other  special  populations.   Based 
on  an  application  of  the  selected 
criteria  (Fig.  1)  to  mostly  used 
cited  existing  sources  of  minority  of 
minority  health  data,  the  following 
limitations  were  identified: 

•  Inadequate/small  numbers  in 
survey  sample,  unstable 
estimates; 

•  Underreporting  of  race  (e.g. 
NHDS) ; 

•  Inappropriate  Sampling  frames, 
lack  of  appropriate  denominator; 

•  Lack  of  data  elements  for 
assessing  nativity,  period  of 
immigration,  and  generation  in 
the  U.S.  ; 

•  Inadequate  data  elements  for 
Socioecomic  Status; 

•  Lack  of  longitudinal  data  for 
planning  and  needs  assessment; 

•  Varied/ Incomplete  collection  of 
race/ethnicity  data  at  some 
State  level. 

Much  of  the  data  elements 
required  for  a  comprehensive 
assessment  of  the  health  needs  of 
minorities  and  other  special 
populations  are  either  not  being 
collected  at  all  by  existing  data 
systems,  some  are  being  collected 
though  by  methodologically 
incompatible  surveys,  or  suffer  from 
lack  of  detailed  information  on 
representative  number  of  minority 
populations  in  their  samples. 

PROPOSED  SOLUTIONS  AND  APPROACHES  FOR 
IMPROVING  ASSESSMENT  OF  MINORITY 
HEALTH  NEEDS 

A  specific  national  objective 
(#22.3)  for  year  2000  is  to  develop 
and  disseminate  among  Federal,  State, 
and  local  agencies  procedures  for 
collecting  comparable  data  and  to 
incorporate  these  procedures  into 
Public  Health  Services  (PHS)  data 
collection  systems.   The  development 
and  dissemination  of  such  comparable 
procedures  for  data  collection  would 
facilitate  comparability  of  data  on 
health  status  within  and  among  State 
and  local  areas  and  would  permit  the 
valid  comparison  of  local  and  State 
health  data  with  national  data. 
(DHHS,HP  2000,  1991) . 


The  following  are  some 
recommendations  for  designing  studies 
and  appropriate  procedures  for 
collecting  data  to  assess  the  health 
needs  of  minorities  and  other  special 
populations.   In  all  instances,  the 
purpose (s)  of  the  data  collection 
should  be  stated  clearly  and 
communicated  to  the  community/ 
populations  of  interest. 

Study  Design (s)  and  Data  Collection 

Data  element  collected  and  sampling 
frames  used  by  existing  data 
collection  mechanisms  often  fail  to 
give  reliable  representation  of 
cultural  diversity.   Current  survey 
use  0MB  directive  15' s  definitions  of 
racial  and  ethnic  populations  in  the 
U.S.  However,  obtaining  specific 
information  about  national  origin 
rather  than  using  an  ethnic  or  racial 
approximations  would  be  more 
informative.   This  type  of  data  yield 
useful  information  especially  when  it 
is  considered  along  with  recency  of 
immigration  or  length  of  residency. 
Surveys  that  inquire  about  health 
care  utilization  should  include  more 
minority  physicians  in  private 
practice  and  traditional  (community) 
healers  in  their  sample  because  these 
are  often  used  by  special 
populations. 
Study  design  should: 

•  Depend  on  the  purpose  of  the 
study  health  issue,  and 
population  of  interest; 

•  Establish  mechanisms  and 
systems  for  longitudinal  data; 

•  Use  appropriate  standard 
populations; 

•  Oversample  more  subgroups 
within  subpopulations; 

•  Involve  minority  populations  in  all 
phases  of  the  project; 

•  Relate  data  elements  to  core 
health  indicators/measures. 

Data  collection  should: 

•  Expand  beyond  minimal 
classification  in  0MB  Directive  15; 

•  Develop  uniform  core  questions  for 
all  new  data  collections; 

•  Establish  procedures  for 
longitudinal  data  collection; 

0  Face-to-face  vs  telephone 
household  survey; 

•  Plan  comprehensive  periodic 
survey  focusing  on  minority 
population  (regional/  community 
based) ; 

•  Establish  panel  of  experts  for 
kinds  of  data  needed; 

•  Ascertain  language  competence  of 
target  population; 

•  Use  and  involve  neighborhood 
(community)  health  centers; 

•  Employ  indigenous  outreach  workers. 
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Data  Analyses 

Several  published  sources  that 
analyze  available  data  do  not  analyze 
the  data  such  that  it  can  be  used  for 
accurate  assessment  of  the  health  of 
special  population.   For  example, 
there  must  be  rationale  behind  age 
groupings. 
Data  analyses  should: 

•  Provide  rationale  for  the  use  of 
race  and  ethnicity/national  origin; 

•  Establish  similarities  before 
grouping; 

•  Provide  rationale  for  age  groupings 
(i.e.,  stages  of  human  development, 
etc . ) ; 

•  Go  beyond  descriptive  analysis  to 
more  multivariate  crosstabulations 
and  regression  analysis; 

•  Perform  more  intra-group  analysis 
(inter-group  comparison  is  still 
needed) ; 

•  More  categorical  data  analysis;  and 

•  Identify  "models"  of  effective 
intervention(regional  and  local 
variations) . 

DISCUSSION 

Much  has  been  said  and  written 
about  "enrollment  form"  and 
"encounter  form"  in  the  context  of 
health  care  reform.   However,  the 
proportion  of  short-stay  hospitals 
not  reporting  race  in  National 
Hospital  Discharge  Survey  (NHDS)  is 
increasing  with  increasing  use  of 
automated  data  collection  systems. 
More  reliable  data  are  needed  than 
medical  care  data  for  comprehensive 
assessment  of  the  health  needs  of 
minorities  and  other  special 
populations.   Standardized  set  of 
data  elements  and  health  status 
indicators  should  provide  information 
about  community  health,  personal 
health,  environmental  and 
occupational  health,  as  well  as 
mental  health  needs  of  the  community. 
The  concept  of  "community  diagnosis" 
as  compared  to  "personal  diagnosis" 
should  be  employed  in  any 
comprehensive  assessment  of  the 
health  needs  of  minority  populations. 

Health  needs  assessment  per  se 
is  useless  without  the  use  of  the 
information  to  plan,  implement,  and 
evaluate  appropriate  health  programs 
needed.        Data  collection  systems 
and  agencies  should  be  encouraged  to 
go  beyond  the  minimal  race  and 
ethnicity  categories  in  OMB  Directive 
15.   In  addition  to  Hispanics, 
Asians,  and  Pacific  Islanders,  the 
heterogeneity  and  increasing  cultural 
diversity  of  blacks  in  the  U.S.  needs 
also  be  reflected  in  future  data 
collection  systems. 


Reliable  baseline  assessment  is 
crucial  for  health  planning  and 
evaluation.   Expanded  State-based 
utilization  of  standardized  health 
status  indicators  and  comparable 
procedures  for  data  collection  will 
enhance  the  appropriateness  and 
utility  of  data  systems  for  assessing 
the  health  needs  of  minorities  and 
other  special  populations  beyond  year 
2000. (DHHS,  NCVHS  1993  Annual  Report)  . 
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INTRODUCTION 

Patient  care  quality,  costs, 
outcome,  and  access  are  primary  issues 
facing  today's  health  care  industry,  its 
reimbursers,  and  consumers.  Currently, 
limitations  of  existing  patient  care  data 
hamper  the  ability  of  researchers  and 
clinicians  to  assess  the  differences  in 
the  way  health  care  is  accessed  by  and 
delivered  to  different  populations  and 
the  impact  of  these  differences  on  the 
outcomes  from  care.  The  study  described 
in  this  paper  identified  extant  Automated 
Medical  Record  Systems  (AAMRS)  containing 
health-related  minority  data.  The  study 
explored  whether  these  data  could  be 
aggregated  to  address  the  following 
research  question:  Are  there 
racial/ethnic  differences  in  the 
treatment  and  outcome  for  patients  with 
hypertension?  In  phase  I  of  the  study 
records  were  identified  and  obtained  from 
AAMRS  to  describe  the  value  and  quality 
of  the  data  to  be  pooled  across  data  sets 
to  address  the  question.  In  phase  II 
analytic  approaches  to  these  data  sets 
were  explored,  described,  and  tested. 

PHASE  I  METHODS 

A  review  of  potential  AAMRS  was 
conducted.  Seven  data  base  owners  were 
identified  and  sent  a  solicitation  letter 
and  short  questionnaire  which  queried  the 
relevance  of  the  data  in  their  data  bases 
to  the  research  question  to  be  answered. 
Of  these  seven  owners,  only  two  were  able 
to  comply  with  the  request,  furnishing 
data,  on  diskette,  within  the  time  frame 
necessary. 

Fulfilling  the  request  was  difficult 
for  data  base  owners  and  data  management 
staff  because  of  the  time  required  and 
the  logistical  problems  involved,  and  the 
financial  and  time  costs  of  data 
retrieval.  In  addition,  technical 
constraints  impeded  acquisition  of  data. 
Many  of  the  data  are  stored  in  data  bases 
designed  for  clinical  care  in  ambulatory 
settings;  to  meet  the  research  needs  of 
this  project,  a  great  deal  of  programming 
and  effort  was  required  to  make  the  data 
suitable . 

DATA  SET  EXPLORATION 

Data  sets  from  University  of 
Nebraska  Medical  Center  (UNMC)  and  Duke 


Medical  Center  (Duke)  ambulatory  settings 
were  selected  for  the  project.  After 
controlling  for  number  of  visits  (>  one), 
age  (45-75  years),  and  primary  diagnosis 
(hypertension,  ICD-9  code  401-404),  1468 
patient  records  from  the  UNMC  data  sets 
were  eligible  and  958  from  Duke  were 
eligible.  Scrambled  identifiers  were 
used  on  all  records  to  protect  the 
privacy  and  confidentiality  of  the 
patients . 

Race/Ethnicity  Data 

Race/ethnicity  descriptors  were 
collected  and  stored  in  both  data  sets. 
The  number  of  minorities,  other  than 
blacks,  represented  in  both  data  sets 
were  small.  (Table  1)  OMB 's  Directive 
15,  which  sets  forth  race  and  ethnic 
standards  for  Federal  Statistics,  was 
used  as  a  comparative  base  and  its 
racial/ethnicity  definitions  were  not 
altered.  (Table  2)  Although  different 
codes  were  used  by  UNMC  and  Duke,  they 
were  matched  and  recoded.  The  Duke  data 
set  was  missing  a  large  number  of 
racial/ethnic  data  and  no  materials  were 
available  to  document  how  the  data  which 
were  available  were  collected  and  by 
whom.  No  distinction  was  made  between 
missing  and  unknown  values.  In  addition, 
in  the  UNMC  data  set,  all  data  coded  for 
ethnicity  were  missing,  but  some 
ethnicity  data  were  found  in  "race." 

Other  Demographic  Data 

Education,  occupation,  and 
socioeconomic  data  are  often  used  to 
complement  race/ethnicity  data  in 
analysis.  These  data  as  well  as  gender, 
age,  and  marital  status  influence  the  use 
of  health  care  services,  barriers  to 
health  care,  and  treatment  outcomes.  As 
important  as  these  data  are  believed  to 
be,  they  were  often  missing  or  incomplete 
in  not  only  the  UNMC  and  Duke  data  sets, 
but  also  in  the  other  data  sets  which 
were  reviewed.  For  example,  only  Duke 
collects  data  on  education  and  even  these 
are  fragmentary  (58%  of  the  values  were 
missing) .  UNMC  does  not  store 
occupational  data  and  the  data  collected 
by  Duke  determine  only  whether  or  not  the 
person  has  an  occupation.  Gender  data 
were  available  on  all  patients  in  both 
data  sets.  Age  was  also  available  in 
both  sets  and  was  either  collected  at  the 
time  of  visit  or  calculated  from  the  date 
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of  birth.   Marital  status  was  available 
for  almost  all  patients  in  each  data  set. 

Medical   History,    Encounter   Related, 
Laboratory  Measures,  and  Treatments 

Medical  history,  encounter, 
laboratory  and  treatment  regimen 
influence  the  relationship-  between 
race/ethnicity  and  the  treatment  of  and 
outcome  from  disease.  Like  demographic 
data,  many  of  these  variables  were  not 
available  in  the  data  sets  reviewed.  Age 
at  onset  was  found  in  only  the  Duke  data 
set,  but  comorbid  diseases  and 
smoking/alcohol  use  were  found  in  both. 
Several  comorbid  diseases  are  applicable 
to  hypertension  and  both  UNMC  and  Duke 
use  ICD-9-CM  codes  to  record  them, 
although  Duke  will  sometimes  use  a  (TMR) 
code  specific  to  their  text.  The 
parameters  for  smoking  and  alchohol  were 
not  clearly  defined,  were  recorded 
unevenly  in  each  data  set,  and  were  often 
missing.  Level  of  exercise  was  collected 
by  UNMC,  but  not  by  Duke;  family  history 
of  hypertension  was  unavailable  in  the 
UNMC  data  set  and  not  typically  used  in 
the  Duke  data  set.  Encounter-related 
data  and  laboratory  measures  were 
contained  in  both  data  sets .  Encounter 
data  varied  slightly  between  UNMC  and 
Duke  while  laboratory  measurements 
appeared  to  be  collected  in  the  same 
format.  Data  on  treatments, 
(pharmaceutical  therapies  and  behavioral 
interventions),  were  collected  unevenly. 
For  this  reason,  a  subset  of  medication 
was  selected  for  analysis. 

DATA  QUALITY  AND  COMPARABILITY 

The  quality  and  useability  of  any 
data  contained  within  a  medical  record  or 
a  data  set  is  influenced  by  the  many 
steps  of  the  data  collection  process. 
Each  physician-patient  encounter 
generates  data  and  creates  the 
opportunity  for  error.  Data  reporting  by 
the  patient,  data  collection  by  the 
provider,  data  capture  (whether  automated 
or  manual),  data  entry  and  data  retrieval 
influence  the  quality  of  the  data. 
Assumptions  about  the  data  are  often  made 
away  from  the  physician-patient 
encounter.  These  are  often  not 
documented  and  influence  the  data.  For 
example,  both  UNMC  and  Duke  data  sets 
contain  a  number  of  "blanks"  and 
available  documentation  does  not  indicate 
whether  these  are  missing  data  or 
unknowns.  It  is  also  unclear,  especially 
in  reference  to  race  and  ethnicity 
categories  and  quality  of  life  data,  how 
determinations  and  classifications  are 
made.  Individual  interpretation  and 
provider/collector  idiosyncracies  will 
thus  influence  the  collection  of  many  of 
these  data.  ■ 


Treatment  outcome  and  quality  of 
life  are  difficult  to  assess  across  data 
sets.  Data  elements  and  collection 
procedures  are  not  standardized  and  do 
not  lend  themselves  to  reprogramming  to 
increase  uniformity.  As  is  seen  in  UNMC 
and  Duke  data  sets,  and  others  reviewed, 
the  process  of  care  is  not  detailed  and 
data  from  any  care  received  in  a 
secondary  setting  or  in  anecdotal 
episodes  are  not  collected. 

The  UNMC  and  Duke  data  sets  were 
often  incomplete.  Lack  of 
standardization,  irregularity  of 
collection,  or  simply  data  not  being 
collected,  all  influenced  the 
comparability  of  the  data  sets  and  the 
ability  to  link  them  to  answer  the 
medical  effectiveness  question. 

VALUE  OF  THE  LINKED  DATA 

Despite  the  fact  that  the  data  were 
not  collected  for  these  purposes  —  to 
link  across  data  sets  and  for  medical 
effectiveness  research,  data  from  the  two 
data  sets  were  "linkable"  in  the  sense 
that  data  collected  in  disparate  health 
settings  could  be  aggregated  at  a  level 
which  allowed  effective  medical 
effectiveness  research.  This  finding  is 
important  to  future  research.  Data 
pooled  in  this  method  allow  a  variety  of 
data  elements  and  fields  to  be  compared 
and  draw  from  a  larger  population  than  is 
available  through  a  single  institution's 
records.  By  addressing  the  data 
deficiencies  noted  in  this  paper,  these 
could  become  excellent  sources  of 
research  data. 

PHASE  II  METHODS 

Patients,  aged  45-75  years,  with  a 
diagnosis  of  hypertension  were  extracted 
from  the  Duke  and  UNMC  data  sets  and 
divided  according  to  OMB 's  Directive  15 
into  two  racial/ethnic  groups  —  "white" 
and  "other  races."  (Table  3)  Outcome 
variables  were  chosen  and  predictor  or 
antecedent  variables  were  defined.  Data 
sets  were  recoded  and  structured  to  allow 
the  data  sets  to  be  analyzed. 

FINDINGS 

The  reporting  of  race  and  ethnicity 
reflect  differences  between  the  Duke  and 
UNMC  data  sets.  The  UNMC  data  set 
contains  race/ethnicity  for  almost  all  of 
the  patients;  however,  48%  of  the 
race/ethnicity  data  is  missing  in  the 
Duke  data  set .  These  records  could  not 
be  included  in  the  main  analysis.  In 
order  to  assess  the  generalizability  of 
the  findings  for  the  remaining  patients, 
a  series  of  chi-squares  was  used  to 
evaluate  differences  in  outcome  variables 
or   explanatory   variables   between   the 
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group  with  missing  race  data  and  the  52% 
of  Duke's  patient  records  remaining  for 
the  main  analysis.  Table  4  shows  the 
findings.  The  patients  with  missing 
race/ethnicity  data  do  not  differ 
significantly  from  the  remaining  group  on 
gender  distribution,  myocardial 
infarction  after  first  visit,  or  whether 
antihypertensive  medications  were  ever 
used.  However,  there  is  a  significant 
difference  (p<.013)  in  the  proportion  of 
patients  hypertensive  at  the  end  of  the 
study;  44%  (n=204)  of  patients  with 
missing  race  data  were  hypertensive  while 
36%  (n=181)  of  those  with  complete  race 
data  were  hypertensive.  There  is  also  a 
marginally  significant  difference 
(p<.085)  in  the  proportion  of  patients 
having  a  stroke  after  first  visit;  3% 
(n=14)  of  patients  with  missing  data  had 
a  stroke  after  first  visit  while  1%  (n=7) 
of  those  with  complete  race  data  had  a 
stroke  after  first  visit. 

While  there  is  a  significant 
difference  (t-test  with  p<.001)  in  the 
mean  ages  (approximately  three  years)  of 
the  patients,  the  clinical  significance 
of  the  difference  is  unclear.  Non- 
significant differences  in  gender,  stroke 
or  myocardial  infarction  after  first 
visit,  and  use  of  antihypertensive 
medication  may  be  clinically  significant, 
but  it  is  not  possible  to  determine  if 
they  are  race/ethnic  related. 

In  comparing  the  Duke  and  UNMC 
populations,  we  found  that  there  is  no 
difference  in  the  age  or  gender  of  the 
two  samples,  but  there  are  differences  in 
all  other  demographic  and  outcome 
variables.  The  patients  at  UNMC  were 
predominantly  "white"  while  at  Duke,  they 
were  evenly  divided  between  "white"  and 
"other  races."  However,  the  Duke  data 
reflect  only  those  patients  for  whom 
there  is  racial/ethnic  data. 

The  relationship  of  race/ethnicity 
to  outcome  was  explored  using  a  logistic 
regression  model.  Table  5  shows  the 
findings.  Race  did  not  influence  the 
occurrence  of  stroke  after  first  visit  or 
myocardial  infarction  after  first  visit. 
Moreover,  when  the  occurrence  of  either 
stroke  or  MI  after  first  visit  were 
evaluated  together,  race/ethnicity  was 
the  only  predictor  shown  to  have  a  non- 
significant effect.  The  presence  of 
hypertension  at  the  end  of  the  study  was 
not  shown  to  be  influenced  by 
race/ethnicity . 

DISCUSSION 

Because  racial/ethnic  data  were 
missing  from  48%  of  the  Duke  data  set, 
this  key  variable  could  not  be  analyzed 
for  the  total  data  set.  Therefore,  those 
patients  with  racial/ethnic  identifiers 


and  those  without  these  identifiers  were 
compared.  Results  indicate  that  these 
two  groups  do  not  differ  on  gender, 
proportion  with  a  myocardial  infarction, 
or  use  of  antihypertensive  medications. 
Analyses  of  other  variables,  such  as 
obesity  or  the  use  of  specific 
antihypertensive  medications  were 
considered.  However,  because  of  the 
paucity  of  data  and/or  the  absence  of 
racial  categories,  the  analyses  would  not 
have  been  meaningful.  While  the  mean 
ages  of  the  patients  in  the  two  groups 
were  statistically  different,  the  3  year 
average  age  difference  does  not  seem 
meaningful  from  a  clinical  standpoint. 
Duke  patients  with  missing  race  data  were 
more  likely  (p<.013)  than  those  with 
complete  racial/ethnic  data  to  be 
hypertensive  at  the  end  of  the  study. 
(Table  4)  Duke  patients  with  missing 
race  data  were  also  marginally  more 
likely  (p<.085)  than  those  with  complete 
racial/ethnic  data  to  have  had  a  stroke 
after  the  first  visit.  (Table  4) 

The  literature  suggests  that  there 
is  often  misclassif ication  of 
racial/ethnic  minorities  but  the  bias 
that  the  misclassif ication  introduces  in 
understanding  differences  in  diagnosis, 
treatment,  and  outcomes  is  likely  to  be 
negligible.  Blustein  found  that 
admitting  clerks  were  most  likely  to 
obtain  and  record  this  information.1  A 
careful  review  of  biomedical, 
epidemiological,  and  research  data 
suggests  that  missing  racial/ethnic  data 
cannot  be  determined  nor  is  there  an  a 
priori  method  for  imputing  the 
information.  The  Census  Bureau  imputes 
missing  race/ethnicity  data  from  other 
family  members,  housemates,  neighbors  and 
the  neighborhood  in  general.2  This 
approach  is  not  generally  applicable  in 
the  case  of  medical  records  where  the 
individual  generally  presents  alone  and 
the  race/ethnicity  of  next  of  kin  is  not 
recorded. 

Little  is  known  about  patterns  of 
missing  race  and  ethnicity  data  in 
patients'  medical  records  and  it  is 
unclear  whether  the  provider  gathering 
the  data,  the  setting  in  which  the  data 
are  gathered,  or  the  actual  race  and 
ethnicity  influence  the  proportion  of 
missing  data  in  these  categories.  We 
recommend  that  this  issue  be 
systematically  studied. 

Some  research  suggests  that  the 
health  seeking  behavior  of  minorities, 
because  of  their  lower  socioeconomic 
status,  lack  of  access  to  care,  or  fear 
of  the  health  care  system  results  in 
their  receiving  care  later  in  the  disease 
process  and  with  less  regularity  than 
non-minorities.3  It  is  possible  that 
their  first  encounter  with  the  health 
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care  system  is  on  an  emergency  basis, 
where  obtaining  demographic  information 
is  not  the  first  concern.  Further 
research  into  the  distribution  of  those 
whose  race/ethnicity  is  classified  and 
those  for  whom  it  is  not  would  shed  light 
on  the  meaning  of  this  missing  data. 

In  order  to  address  the 
racial/ethnic  question  posed  for  this 
study,  comparison  across  the  Duke  and 
UNMC  data  sets  included  only  those  Duke 
patients  (n=497)  whose  race/ethnicity  was 
recorded  on  the  patient  record.  The 
elimination  of  patients  without 
racial/ethnic  identifiers  hinders  the 
generalizability  of  the  findings; 
however,  it  did  enable  analysis  of  the 
data . 

Patients  at  UNMC  who  had  race  data 
were  predominantly  "white",  while  at 
Duke,  they  were  evenly  divided  between 
"white"  and  "other  races."  It  appears 
that  UNMC  patients  were  "sicker"  at  the 
end  of  the  study;  that  is,  were  more 
likely  to  have  myocardial  infarction  or 
stroke,  and  more  likely  to  be 
hypertensive  at  the  end  of  the  study. 
This  may  reflect  patient  status  or,  in 
fact,  more  complete  record  keeping  on  a 
population  who  regularly  seeks  care  at 
the  same  facility.  It  is  interesting  to 
note  that  this  population  at  UNMC  is  more 
likely  to  have  been  prescribed 
antihypertensive  medication. 

The  contribution  of  individual 
predictors  to  the  outcome  from  treatment 
was  assessed.  Stroke  after  first  visit 
and  hypertensive  at  end  of  study  were 
associated  with  site,  as  described  above. 
Stroke  was  more  likely  to  occur  in 
patients  who  used  antihypertensive 
medications .  Because  stroke  is  often 
associated  with  continued  elevated  blood 
pressure,  it  is  not  surprising  that  these 
two  associations  occur  together. 
Perhaps,  it  is  the  "sickest"  patients, 
the  ones  with  most  complications,  who 
receive  medication  and  therefore, 
antihypertensive  use  is  also  associated 
with  site.  Myocardial  infarction  after 
first  visit  was  associated  with 
antihypertensive  medication  use,  and  also 
with  site  and  gender.  When  combined, 
that  is,  stroke  or  myocardial  infarction 
after  first  visit,  all  predictors  except 
race  were  significant.  Moreover,  the 
significance  of  race/ethnicity  does  not 
appear  to  be  meaningful  (22.7%  whites  vs. 
21.7%  blacks)  in  the  combined  analyses, 
and  does  not  play  a  significant  part  in 
predicting  either  event  separately. 

Reasons  for  these  differences  are 
not  clear.  The  literature  suggests  that 
black  patients,  who  in  this  study 
represent  the  majority  of  the  "other 
races"   population,   are   likely  to  be 


sicker  and  less  compliant  than  white 
patients.4  While  UNMC  appears  to  have  a 
greater  proportion  of  white  patients  who 
are  "sicker",  much  of  the  racial/ethnic 
data  (48%)  is  missing  from  the  Duke  data 
set  and  comparisons  cannot  reasonably  be 
made.  Differences  in  collection,  entry, 
and  retrieval  of  data  may  also  contribute 
to  the  difference.  Or  in  fact,  patients 
at  UNMC  may  be  sicker  when  they  present 
for  treatment.  Duke  may  follow  patients 
more  aggressively  and  make  more  referrals 
for  behavior  modification  of  risk 
factors,  thereby  using  fewer  medications 
and  showing  better  outcomes  from 
treatment.  Because  these  data  were  not 
available  in  the  data  sets,  no  definitive 
evaluation  can  be  made. 

CONCLUSIONS  AND  RECOMMENDATIONS 

Because  of  the  differences  in  racial 
distribution  and  the  dissimilarities  of 
the  sample  population  in  the  two  data 
sets,  only  a  portion  of  the  data  could 
reasonably  be  pooled  to  address  a  medical 
effectiveness  research  question. 

Little  is  known  about  the  meaning  of 
missing  racial/ethnic  data.  It  is 
unclear  whether  it  reflects  minority 
status  or  is  influenced  by  the  provider 
collecting  the  data  or  the  setting  in 
which  the  data  are  gathered.  Training  of 
providers  for  systematic  racial/ethnic 
data  collection  may  result  in  more 
reliable  data. 

Further  research  is  needed  to  assess 
the  role  of  race/ethnicity  in  the 
identification,  treatment,  and  outcome 
from  hypertension.  Efforts  must  be  made 
to  understand  and  document  the  way  in 
which  racial/ethnic  data  are  collected, 
by  whom,  and  with  what  assumptions.  The 
meaning  of  missing  racial/ethnic  data 
must  be  explored.  Based  on  the 
characteristics  of  the  data  sets  explored 
in  this  study,  we  would  not  a  priori 
recommend  aggregating  data  across  care 
centers.  As  shown,  each  of  the  potential 
data  sets  must  be  explored  and  defined 
and  the  differences  assessed  with  respect 
to  the  research  questions  posed.  If 
researchers  can  use  AAMRS  care  data  to 
perform  medical  effectiveness  research, 
they  must  be  cognizant  of  its  limitations 
in  both  content  and  quality  and 
incorporate  these  limitations  into  their 
findings . 
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TABLE  1:  RACE/ ETHNICITY  OVER  POOLED  DATA 
(OMB  DIRECTIVE  15) 


Blank 

Am  Ind 
Alaskan 

Asian/ 
Pac  Is 

Af  Am/ 
Black 

Cauc/ 
White 

Hispanic 

Total 

DMC 

461 

1 

5 

237 

253 

1 

958 

NMC 

10 

11 

12 

363 

1062 

10 

1468 

Total 

471 

12 

17 

600 

1315 

11 

2426 

Note:    DMC  =  Duke  University  Medical  Center;  NMC  =  Nebraska  University 
Medical  Center. 


TABLE  2:   OMB  DIRECTIVE  15 
RACE/ETHNICITY  CATEGORIES 


American  Indian  or  Alaskan 
Native 


Asian/Pacific  Islander 


African  American/Black 


Caucasian /White 


Hispanic — 1  ethnic  group 


TABLE  3:  RACE/ETHNICITY  OVER 

POOLED  DATA 

(OMB  DIRECTIVE  15  "REDUCED  FORM") 


Blank 

Cauc/ 

White 

Other 
Races 

Total 

DMC 

461 

253 

244 

958 

NMC 

10 

1062 

396 

1468 

Total 

471 

1315 

640 

2426 

TABLE  4:  DUKE  -  PRESENCE  OF  RACIAL/ETHNIC  DESCRIPTORS  * 


No  R/E  Data 

R/E 

Data 

x2 

P 
value 

N 

% 

N 

% 

Hypertensive  at  End 
of  Study 

6.  105 

0.013 

No 

257 

56% 

316 

64% 

Yes 

204 

44% 

181 

36% 

Stroke  after  1st 
Visit 

2.958 

0.085 

No 

447 

97% 

490 

99% 

Yes 

14 

3% 

7 

1% 

*  only  significant  and  marginally  significant  results  are  shown  due  to  space  limitations 
TABLE   5:  OUTCOME  AFTER  FIRST  VISIT  OR  AT  END  OF  STUDY  * 


Predictor 

A  ** 

p  value 

B  *  * 
p  value 

c  ** 

p  value 

D  ** 
p  value 

Site 

0.001 

0.000 

0.000 

0.000 

Gender 

NS 

0.016 

0.004 

NS 

Race/ 
Ethnicity 

NS 

NS 

NC 

NS 

Antihy . 
Medication 

0.000 

0.000 

0.000 

0.003 

*  only  significance  of  results  reported  due  to  space  limitations 

**  A  =  Stroke  after  first  visit;  B  =  MI  after  first  visit; 

C  =  Stroke  or  MI  after  first  visit;  D  =  Hypertensive  at  end  of  study 
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Introduction 

Focus  group  research  has  become 
a  widely  accepted  in  and  often 
integral  to  many  types  of  public 
health  inquiry.  These  range  from  the 
formative  phases  of  health  promotion 
research  to  testing  educational 
messages  and  materials (Ramirez , 
1988;  Basch,  1987),  and  the 
development  of  health  survey 
questions  (O'Brien,  1993) .  In  the 
course  of  our  research  on  the  cross- 
cultural  comparability  of  survey 
methods  and  findings1,  we  discovered 
a  surprising  void  in  the  literature 
regarding  the  appropriateness  of 
focus  group  methods  across  cultures. 

Focus  groups  are  small  group 
discussion  sessions  among  members  of 
a  study  or  program  target 
population.  Participants  are  guided 
through  a  standardized  series  of 
open-ended  questions  by  a  trained 
moderator  to  elicit  individual 
responses.  In  this  way,  concepts  can 
be  generated,  hypotheses  explored, 
and  idiomatic  expressions 
identified.  Over  the  past  decade,  a 
considerable  body  of  literature  has 
been  produced,  yielding  generally 
accepted,  standardized  focus  group 
procedures  (Krueger,  1994;  Goldman 
and  McDonald,  1987)  .  The  ubiquitous 
use  of  this  qualitative  tool  has 
elevated  the  importance  of  focus 
group  methods  and  findings  in  public 
health  policy  and  programs.  Thus, 
the  lack  of  thoughtful  reflection  on 
the  cultural  appropriateness  of 
focus  groups  and  their  methodologic 
components  has  far-reaching 
implications.  As  a  first  step 
toward  addressing  this  problem,  we 
convened  a  panel  discussion  to 
explore  the  unique  and  important 
cultural  issues  that  arise  when 
conducting  focus  groups  with 
specific  race/ethnic  groups . 

Purpose  and  Approach 

Our  study  of  the  comparability 
and  validity  of  health  survey 
methods  and  findings  across 
race/ethnic  groups  ,  languages,  and 
cultures  included  a  series  of  focus 
groups  in  African  American,  Chinese, 

Hispanic,  and  Vietnamese 
communities.  However,  in  planning 
these  groups  intended  to  explore  the 
cross-cultural  appropriateness  of 
quantitative  methods,  we  came  to 
also  question  whether  or  not  this 
qualitative  approach  can  in  fact  be 
used  in  comparable  ways  across 
race/ethnic  groups .  A  review  of  the 
literature  revealed  no  prior 
consideration  of  this  issue  leaving 
us  with  the  following  questions.  Are 
state  of  the  art  conventions  in 
focus  group  methodology  applicable 
uniformly  across  cultures  and 
topics?  Do  researchers  and 
moderators  who  conduct  focus  groups 
with  members  of  their  own 
race/ethnic  group  intuitively  or 


explicitly  adapt  these  methods  to 
characteristics  of  the  culture? 

While  many  research  reports 
briefly  mention  the  use  of  focus 
groups  with  specific  race/ethnic 
groups  as  a  component  of  their 
study,  comments  are  limited  to  the 
simple  fact  that  groups  were  used, 
not  how.  In  this  information  vacuum, 
myths,  stereotypes,  and  hunches 
abound:   "You  can't  do  focus  groups 
with  Asians,  because  they  are  so 
reticent  to  speak  among  strangers;" 
and  "you  can't  do  them  with  American 
India.'.  3  because  of  distrust";  "focus 
groups  on  sensitive  topics  can't  be 
done  with  Hispanics  because  there  is 
a  cultural  reserve  that  would  make 
this  unthinkable,-"  and  "the  same 
group  must  be  re-convened  multiple 
times  to  elicit  personal  information 
from  Filipinos,  because  they  require 
an  extended  time  period  to  establish 
trust."  The  first  three  comments 
mentioned  above  have  been  refuted 
through  our  own  work  and  many  other 
studies.  The  last  is  potentially 
valid,  and  is  currently  under 
investigation  } 

Because  so  much  is  being  done 
with  focus  groups  on  different 
issues  and  with  diverse  populations, 
we  believed  that  much  could  be 
learned  about  cultural 
appropriateness  of  the  methods  by 
mining  the  knowledge  and  experience 
of  individuals  who  are  skilled  in 
the  conventional  methods ,  and  who 
have  conducted  groups  in  their  own 
race/ethnic  community.  For  maximum 
dialogue,  the  format  of  a  day-long 
panel  discussion  was  chosen  with  two 
experts3  from  each  of  the  four 
populations  who  are  the  focus  of  our 
current  NCHS  and  other  related 
studies  (African  American,  Chinese, 
Hispanic  and  Vietnamese)  .  The 
backgrounds  of  the  panelists 
included  public  health  and  private 
sector  market  research. 

Standard  components  of  focus 
group  research  provided  the 
framework  for  a  moderated  discussion 
consisting  of  panel  dialogue  and 
interaction  with  a  multi-ethnic 
audience  of  about  3  0  of  our  research 
colleagues.  Expert  panelists  were 
asked  to  provide  culturally  relevant 
examples  and  insights  from  their 
experiences  related  to:  project 
design  and  planning  (including 
definition  of  research  questions  and 
participant  characteristics)  ; 
specification  of  respondent 
recruitment  methods ;  discussion 
guide  development,  through  which  the 
questions  to  be  asked  of  the  group 
are  written  and  organized; 
definition  of  moderator 
characteristics  and  techniques; 
planning  for  the  logistics  of  the 
group;  and  the  analyses  and 
interpretation  of  findings.  The  day 
concluded  with  summaries  and 
responses  from  a  multi -ethnic  team 
of  research  assistants  who  had  been 
trained  to  conduct  groups  and  had 
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done  so  prior  to  the  panel 
discussion . 

Insights  from  the  Panel 

Project  design  and  planning . 
Panelists  were  asked  if  there  are 
certain  topics  that  are  best  not 
explored  using  the  group  interview 
format .  There  were  clear  variations 
in  the  responses.  For  example,  one 
of  our  Hispanic  experts  indicated 
that,  once  a  person  agrees  to 
participate  and  actually  shows  up, 
they  will  be  "open,  and  honored  to 
speak" .  Any  reticence  would  come  up 
at  the  time  of  recruitment.  However, 
our  African  American,  Vietnamese, 
and  Chinese  panelists  agreed  that 
"self  disclosure  or  disclosure  about 
one's  family  is  foreign;"  that  very 
sensitive  topics  might  not  be 
discussed  openly  until  a  great  deal 

of  trust-building  occurred,  and  this 
probably  could  not  happen  in  one 
session.  This  implies  the  need  for  a 
departure  from  the  convention  that 
focus  group  participants  should  not 
be  known  to  one  another,  in  order  to 
permit  greater  openness.  Instead  the 
indication  is  that,  for  some 
cultures,  participants  may  require 
greater  familiarity  to  build  trust 
with  each  other  and  the  moderator. 

This  raises  a  question  that  was 
not  resolved  by  the  panel:  If 
considerable  time  is  needed  to 
develop  trust,  would  it  be  better 
initially  to  bring  together  people 
who  are  friends  or  family,  or  would 
that  simply  evoke  a  different  set  of 
problems?  In  fact,  the  Vietnamese 
and  Hispanic  panelists  had  very 
positive  experiences  with  groups  of 
people  known  to  one  another,  and  it 
was  suggested  that  researchers 
convene  both  types  of  groups  to 
assess  the  impact  of  familiarity. 

The  issue  of  trust  was 
repeatedly  raised  during  the  panel 
discussion,  particularly  with  regard 
to  the  importance  of  clearly  stated 
purpose  of  the  research.  It  was 
considered  essential  for  the  study 
team  to  be  open  and  clear  with  the 
community  about  their  reasons  for 
conducting  research.  Also,  it  was 
said  that  participants  should  be 
informed  of  the  ways  in  which  their 
input  would  be  used. 

The  composition  of  a  focus 
group  is  another  important  design 
issue.  Planners  have  been  known  to 
select  participants  based  strictly 
on  race  or  ethnicity,  without 
consideration  of  cultural 
variability  within  groups.  All  of 
our  panelists  pointed  out  variations 
in  their  communities  that  could 
affect  the  dynamics  of  the  group  and 
the  generalizability of  findings. 
For  example,  for  immigrant  groups, 
there  is  heterogeneity  due  to 
acculturation  such  that  persons  from 
opposite  ends  of  the  acculturation 
spectrum  may  not  relate  well  to  one 
another.  For  all  groups, 
educational,  regional,  gender,  or 
even  political  differences  can  be 
important  in  this  way,  depending  on 
the  topic.  Most  of  all,  socio- 
economic differences  figure 
prominently  in  all  aspects  of  group 
design,  implementation  and 


interpretation. 

Ultimately,  all  panelists 
agreed  that  the  degree  of 
segmentation  (selection  of 
homogeneous  population  sub-groups) 
or  mixing  of  sub-groups  depends  very 
much  on  the  topic  of  discussion. 
Several  panelists  mentioned  that  men 
and  women  should  not  be  asked  to 
discuss  sensitive  topics  together; 
in  general,  it  may  be  inadvisable  to 
combine  education  or  literacy 
levels;  and  generations  should  not 
be  intermingled  for  discussion  of 
certain  topics  such  as  sexual 
practices . 

Recruitment .  Recruitment  of 
participants  is  central  to  WHO  you 
get,  and  therefore  WHAT  you  will  get 
OUT  of  the  group.  How  to  recruit 
evoked  some  interesting  differences . 
It  was  noted  that  low  socio-economic 
status  often  negatively  affects  the 
ability  of  people  to  fulfill  their 
commitment  to  attend  a  focus  group, 
so  more  over-recruitment  may  be 
needed,  ranging  from  50%  to  two  to 
three  times  the  intended  number .  The 
convention  for  optimal  size  of  a 
group  is  eight  to  10  participants. 

Method  of  recruitment  also 
varied  because  of  cultural 
differences.  It  was  suggested  that 
the  best  way  to  recruit  Vietnamese 
respondents  is  through  chain 
referral  by  people  who  know  one 
another,  to  benefit  from  a  community 
norm  referred  to  by  one  panelist  as 
"If  you  go,  I  go."   Chain  referral 
was  also  acceptable  to  Chinese  and 
Hispanic  panelists.  Other  methods 
suggested  for  African  Americans  and 
Vietnamese  included  the  use  of  radio 
for  credibility,  and  recruitment 
through  trusted  community-based 
organizations.  The  telephone  was 
rejected  as  a  recruitment  method  for 
Hispanics  and  Vietnamese,  but 
considered  acceptable  for  Chinese. 
Offering  incentives  in  the  form  of 
payment  was  deemed  desirable  across 
all  groups.  Providing  culturally 
appropriate  food  was  emphasized  as 
very  important  to  comfort  and 
rapport  among  respondents . 

Discussion  guide  development . 
It  is  the  convention  to  introduce 
the  topic  and  participants  as 
expediently  as  possible,  explain 
that  the  purpose  is  to  get  their 
opinions,  and  then  move  to  the 
substance  of  the  discussion.  For  all 
the  groups  represented  on  our  panel, 
this  issue  marked  another  departure 
from  standard  methodology.  Trust- 
building  was  identified  again  as  a 
significant  agenda  item  that 
requires  careful  attention,  planning 
and  TIME.  The  description  of  the 
purpose  is  paramount  because  the 
entire  concept  of  the  focus  group  is 
unknown  to  most  people. 

It  was  also  recommended  that  at 
least  thirty  minutes  out  of  the 
usual  one  and  one  half  to  two  hours 
is  needed  for  socializing  among 
Hispanics  .  One  of  the  African 
American  panelists  said  that  the 
first  two  hours  should  be  devoted  to 
building  trust,  using  the  remaining 
third  hour  for  the  "real"  issues.  In 
fact,  it  was  said  that  some  of  the 
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most  important  discussion  occurs 
after  the  formal  conclusion  of  the 
group,  once  the  recording  devices 
are  turned  off.   At  that  point, 
people  have  become  very  comfortable 
and  are  speaking  more  openly  and 
less  self-consciously.  This 
behavior,  as  well  as  many  other 
characteristics  of  focus  group 
dynamics,  was  evidenced  by  our  own 
panel . 

Some  panelists  expressed 
concern  regarding  the  structure  of 
focus  group  questions.  It  was 
reported,  for  example,  that 
Vietnamese..  Chinese,  and  African 
Americans  are  all  "high  context" 
cultures.  This  means  that  questions 
must  be  embedded  in  a  meaningful 
context  or  storyline  that  people  can 
relate  to.  Also,  especially  among 
African  American  and  Chinese,  non- 
verbal communication  is  very 
important.  For  all  groups,  the 
moderator  must  be  able  to  interpret 
this  and  probe  or  respond 
appropriately . 

Moderator  characteristics .  Some 
of  the  most  lively  discussion  was 
generated  by  the  topic  of  moderator 
characteristics  and  skills, 
particularly  whether  or  not  it  is 
necessary  to  have  an  "ethnic  match" 
between  moderator  and  participants. 
All  panelists  agreed  that  the 
cultural  competence  of  the  moderator 
is  paramount,  and  that  a  simple 
physical  match  is  not  sufficient. 
While  they  were  not  asked  to  derive 
a  concise  definition  of  cultural 
competence,  the  issues  raised  by 
panelists  dealt  with:   the 
capability  to  generate  trust  and 
mutual  respect  through  sensitivity 
to  non-verbal  cues;  an  understanding 
of  the  different  culturally- 
influenced  roles  that  participants 
might  assume;  and  knowledge  of  how 
to  properly  respond.  Good 
moderating  skills  are  determined  by 
the  ability  to  moderate  within  a 
specific  culture.  The  example  was 
given  of  an  African  American  group 
in  which  there  may  be  a  participant 
fulfilling  the  role  of  the  "grio", 
the  traditional  story-teller.  A 
culturally  competent  moderator  would 
be  able  to  acknowledge  this  role  and 
incorporate  it  into  the  topic  under 
discussion  by  making  the  relevant 
connections  between  the  personal 
story  and  the  discussion  topic, 
without  allowing  the  story  to 
dominate  or  distract  the  group. 

Vietnamese  and  Chinese 
panelists  all  agreed  that  an  ethnic 
match  was  important  for  their 
communities,  and  that  congruence  in 
language  was  essential.  However,  it 
was  mentioned  that  among  Vietnamese, 
a  young  moderator  might  not  be 
respected  by  either  old  or  young 
participants.  For  the  Hispanic  and 
African  American  panelists ,  there 
was  provisional  acceptance  of 
moderator  diversity.  For  example,  it 
was  suggested  that  a  skilled  white 
moderator  could  function  well  with 
older  African  Americans,  although 
this  might  not  be  the  case  with 
younger  age  groups.  Our  Hispanic 
panelists  disagreed  with  one  another 
on  this  matter.  One  panelist  stated 


that,  for  older  participants,  the 
moderator  should  be  Hispanic  .  The 
other  panelist  felt  that  a  non- 
Hispanic  moderator  with  good  skills 
could  be  effective  regardless  of  age 
group.  For  all  groups,  familiarity 
and  facility  with  the  vernacular  was 
said  to  be  very  important. 

It  was  suggested  by  one 
panelist  that  if  you  don't  have 
enough  moderators  to  match 
respondent  culture  and  ethnicity, 
then  you  should  "grow  your  own" . 
This  is  what  we  did  in  our  own 
study,  by  hiring  and  rigorously 
training  a  team  of  research 
assistants  from  the  four  ethnic 
groups  previously  mentioned. 

Logistics .  There  was  general 
agreement  among  panelists,  regarding 
logistic  arrangements  for  focus 
groups.  The  message  conveyed  was 
that  it  is  best  to  go  to  the 
community,  to  familiar  surroundings, 
for  all  of  these  populations.  One 
Hispanic  moderator  indicated  that 
the  best  group  she  ever  conducted 
took  place  in  a  minister's  living 
room  with  people  who  knew  one 
another.  This  approach  challenges 
the  conventional  practice,  among 
those  researchers  who  can  afford  to 
do  so,  of  holding  groups  in 
professional  facilities  with  hidden 
microphones  and  one-way  mirrors. 
There  were  some  culturally- specific 
logistical  issues  raised,  including 
providing  child- care  for  older 
African  American  women  who  may  be 
responsible  for  their  grandchildren; 
or  holding  groups  during  the  day  for 
Hispanic  women,  and  at  night  for 
Hispanic  men.  Socio-economic 
differences  affect  these  as  well  as 
other  logistic  issues. 

Analyses  and  interpretation.  On 
the  subject  of  analyses  and 
interpretation,  there  was  general 
agreement  that  focus  group  findings 
and  conclusions  are  highly 
subjective  and  thus  readily 
influenced  by  one's  "cultural  lens". 
It  was  therefore  recommended  that 
multiple  analysts  review  the  data 
including  those  with  diverse 
perspectives  from  within  and  outside 
the  culture.  Panelists  cautioned 
against  the  over-generalizationof 
findings  to  "all  Hispanics",  "all 
African  Americans" ,  "all  Vietnamese" 
or  "all  Chinese"  simply  due  to  the 
race  or  ethnicity  of  participants. 

Cross-Cutting  Themes 

Clearly,  the  impressions  of 
eight  individuals  cannot  be 
interpreted  as  valid  or  reliable 
principles.  Like  the  findings 
generated  through  focus  groups,  the 
issues  and  conclusions  raised  in 
this  gathering  are  not  definitive  or 
broadly  generalizable .  However,  as 
is  also  true  for  focus  group 
findings,  the  information  gleaned  is 
very  meaningful  as  a  basis  for 
further  research,  and  potentially 
indicative  of  real  cross-cultural 
similarities  or  culturally-specific 
differences . 

The  dynamic,  spirited,  and 
thoughtful  nature  of  this  panel 
discussion  is  best  conveyed  through 
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the  cross-cutting  themes  that 
emerged  from  the  day.  These  themes 
address  the  issues  of  purpose, 
assumptions,  heterogeneity  and 
cultural  competence  in  the  context 
of  focus  group  methodology.  The 
most  recurrent  theme  was  emphasis  on 
the  importance  of  a  clearly-stated 
purpose   for  focus  groups,  since  this 
affects  every  element  of  the 
research,  from  recruitment  methods 
and  segmentation  to  moderator 
characteristics  and  the  design  of 
the  focus  group  questions. 

Panelists  consistently  warned 
of  the  hazards  in  erroneous 
assumptions,   unidentified  and 
unchallenged  stereotypes  ,  and  over- 
generalization  . 

It  was  clear  that  our  panelists 
were  very  sensitive  to  the 
implications  of  heterogeneity, 
particularly  in  socioeconomic 
status,  in  every  aspect  of  focus 
group  research.  They  also  perceived 
a  strong  interaction  between  the 
purpose  of  research  and  the 
diversity  within  communities. 
Virtually  everything  about 
conducting  the  groups  will  be 
different  with  affluent,  educated 
populations  as  compared  with  those 
who  are  poor  and  less  educated. 

Finally,  all  panelists  agreed 
that  the  cultural  competence  of 
researchers  and  moderators  is  key  to 
the  validity  of  findings.  This 
quality  is  not  simply  a  matter  of 
technique.  Cultural  competence 
demands  sensitivity  to  and  respect 
for  the  individual . 

Preliminary  Conclusions 

From  an  overview  of  the  entire 
day,  we  have  drawn  several 
preliminary  conclusions  .  They  are 
preliminary  because,  again,  this  was 
one  day  of  discussion  with  eight 
professionals.  Their  insights  are 
grounded  in  extensive  experience, 
and  as  such  can  be  viewed  as  sound 
hypotheses  meriting  further  testing 
rather  than  definitive  findings. 

The  panel's  experience 
indicates  that  focus  group  research 
appears  to  be  a  valid  and 
appropriate  methodology  in  the  four 
groups  discussed.  Some  conventions 
such  as  the  single  session  group  may 
be  inappropriate  for  some  topics  in 
some  cultures;  and  some  other 
conventions,  such  as  the  posing  of 
questions  in  a  discussion  group 
format  to  elicit  opinions,  appear  to 
apply  across  all  these  groups 
(although  how  questions  are  posed 
may  vary)  .  According  to  our 
panelists,  cultural  issues  are 
particularly  important  in  moderator 
selection  and  training,  recruitment 
and  segmentation  of  participants, 
and  in  facilitation  techniques . 

This  focus  group  panel 
experience  has  generated  a  long  list 
of  unanswered  questions  that  warrant 
further  research.  Some  examples  from 
the  list  include: 
l.How  does  purpose  affect 
segmentation  and  does  this  vary  by 
culture? 
2.  How  can  diversity  be  represented? 


3 .  What  is  the  relevance  of 

age /genera t ion/gender differences 
in  various  race/ethnic  groups? 

4.  Is  discussion  of  sensitive  topics 
culturally  inappropriate  for  some 
groups? 

5. What  is  optimal  versus  acceptable 
in  moderator  characteristics,  and 
does  this  vary  by  culture? 

The  implications  of  research 
addressing  these  questions  will  be 
felt  throughout  public  health  and 
beyond,  to  other  social  sciences. 
This  is  merely  the  beginning  of  a 
needed  dialogue  and  a  call  to 
research.  We  strongly  encourage 
further  investigations  of  the 
similarities  and  differences  across 
cultures  in  focus  group  research. 

Endnotes 

1.  Improving  Health   Surveys   for  Multi- 
Ethnic   Populations   is  a  grant  to  the 
Northern  California  Cancer  Center  (NCCC) 
from  the  National  Center  for  Health 
Statistics  (NCHS) . 

2.  Early  Cancer  Detection   for  Filipino 
Americans   is  a  grant  to  the  NCCC  from 
the  Department  of  Defense. 

3.  We  would  like  to  acknowledge  the 
panel  participants:  Marcia  Canton,  Lei 
Chun  Fung,  Nichelle  Nickols,  Bang 
Nguyen,  Regina  Otero-Sabogal,  Amelie 
Ramirez,  Chuoc  VoTa,  and  Sandra  Wong; 
Research  Assistants:  Maria  Arrechiga, 
Quita  Bingham,  Jaclyn  Chan,  Thao  Le, 
Ququan  Liu,  Mary  Lou  Munguia,  Thiettranh 
Tran  Pham,  and  Leah  Vaughan;  and  Barbara 
Marin  for  her  presentation. 
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FILLING  IN  THE  BLANKS:  CREATING  AND  AGGREGATING 
SERVICE  DRIVEN  DATA  TO  DEVELOP  COMMUNITY  PROFILES  OF  SPECIAL  POPULATIONS 

Debbie  Indyk,  Dept .  of  Community  Medicine,  Mount  Sinai  Medical  Center 

Len  Indyk 


The  confluence  of  poverty,  AIDS, 
tuberculosis,  substance  abuse  and 
violence  in  our  inner  city 
neighborhoods  makes  them  key  sites  for 
reaching,  assessing  and  developing 
approaches  to  prevention  and  early 
intervention  among  vulnerable  and  hard- 
to-reach  special  populations  who  are 
normally  beyond  the  purview  of  the 
medical  system  until  very  late  in  the 
progression  of  the  multiple  diseases  to 
which  they  are  inequitably  exposed. 

The  inner-city  predominantly 
Latino  and  African  American  inner-city 
neighborhood,  East  Harlem,  served  by 
the  designated  AIDS  Center  of  the  Mount 
Sinai  Medical  Center,  provided  such  a 
strategic  setting  in  which  to  develop 
networks  and  nodes  as  loci  for  service 
delivery,  prevention,  education  and 
research.  The  neighborhood  encompasses 
a  critical  mass  of  individuals  "exposed 
to"  problems  to  be  studied  and  a 
critical  mass  of  "experts"  who  treat, 
study  and  experience  the  problem.  The 
following  two  Tables  of  neighborhood 
demographics  demonstrate  the  range  of 
problems  faced  by  the  community  and  the 
severity  of  the  impact  of  AIDS  on  the 
community. 


TABLE  1 


SELECTED  HEALTH  STATISTICS 
NEIGHBORHOOD 


No  prenatal  care 

18% 

Births-single  women 

73% 

Births-women  on  medicaid 

62% 

Live  births-teen  mothers 

15% 

Low  birth-weight  deliveries 

14% 

Births-drug-using  mothers 

27% 

Child  abuse  and  neglect 

13% 

*  *  -k  "k 

Infant  mortality           17/1 

000 

Tuberculosis             91/100 

000 

Cases -gonorrhea         430/100 

000 

Cases-syphilis          505/100 

000 

TABLE  2 


SELECTED  AIDS  STATISTICS 
NEIGHBORHOOD 


Percent  total  NYC  population  1.4' 
Estimated  total  NYC  HIV  cases  13.5- 
Deaths  attributed  to  AIDS  11  ; 
Latino  &  African  American  87  - 
Percent  total  NYC 

pediatric  AIDS  5  '■ 

AIDS  cases  related  to  IDU  68  '■ 
Estimated  seropre valence 12   s 


The  Aaron  Diamond  Foundation  AIDS 
Prevention  Project  to  the  Department  of 
Community  Medicine  provided  the  engine 
for  sustained  infrastructure  building 
activities  that  fostered  the 
development  of  such  nodes.  Table  3 
shows  the  variety  of  settings  in  which 
the  approach  has  been  applied. 

TABLE  3-  TYPES  OF  SETTINGS 


Family  health  service  agency 
providing  "one  stop  shopping" 

Community  health  service  for 
AIDS/HIV  counselling 

Street -based  needle  exchange 
program 

Family- life  program  providing 
preventive  services  for  high- 
risk  families 

Hospital-based  AIDS  Center 
providing  inpatient  &  outpatient 
care 

Directly  observed  therapy 
(D.O.T.)  program  for 
tuberculosis 

Directly  observed  preventive 
therapy  (D.O.P.T.)  program  for 
tuberculosis 

Hospital-based  employee  health 
service . 


The  approach  provides  sustained 
support  of  resource  program,  staff  and, 
most  importantly,  the  capacity  to 
organize,  aggregate  and  analyze  data  to 
describe  and  assess  the  populations 
they  serve  and  the  work  they  do.  As 
each  agency  elaborates  its  unique 
community  based  or  family-centered  care 
model  and  further  develops  its  data- 
collection  and  analysis  capabilities, 
the  consultants  help  staff  and 
administration  move  their  work  from 
case  study  analysis  and  observational 
studies  to  aggregate  data-analysis. 

The  linchpin  of  the  approach 
involved  the  implementation  of  a 
powerful,  but  easy  to  use  PC-based  MIS 
system  which  (1)  permits  agency- 
specific  planning  and  evaluation  of 
program  activities,  and  (2)  is  critical 
as  a  first  step  in  collecting  and 
analyzing   community-generated   data. 


398 


Since  the  data  collected  are  service- 
driven  and  include  a  great  variety  of 
non-reimbursable  services,  the  needs 
and  utilization  profiles  which  we 
derive  for  these  populations  are  quite 
different  from  those  which  can  be 
obtained  from  the  traditional  sources, 
namely,  hospital  discharge  data  and 
medical  insurance  data. 

The  data-collection,  analysis  and 
reporting  requirements  of  Community- 
Based  Organizations  and  other  types  of 
stand-alone  health-related  facilities 
in  the  USA  in  the  1990 's  have  become 
extremely  complex.  Most  often,  however, 
agencies  develop  systems  which  allow 
them  only  to  collect  those  data 
required  by  their  f undors .  The  various 
data  systems  are  not  linked  and  do  not 
give  the  agencies  the  ability  to  fully 
describe  who  it  is  they're  serving,  for 
how  long  and  with  what  outcome.  The  MIS 
system  which  we  developed  gives  CBO's 
capability  and  flexibility  which  rivals 
that  of  a  Medical  Center's  Data 
Processing  department . 

The  methodology  presented  is 
applicable  to  a  great  variety  of  small 
organizations  in  many  different  types 
of  practice  settings.  All  of  them 
currently  face  a  similar  set  of 
requirements  and  challenges  (Table  4) . 


TABLE  5A  -  HOME  BASED  SERVICES 


TABLE  4 


1995 


CHALLENGES  TO  A  CBO 


1.  The  continually  changing  needs 
of  the  community  and  agency 

2 .  Limited  resources  which  must  be 
maximized  in  an  economy  of 
scarcity 

3.  Multiple  funding  streams  with 
very  different  (but  overlapping) 
reporting  requirements 

4 .  The  impending  and  accelerated 
implementation  of  managed  care 

5.  The  reality  that  CBOs  are  often 
at  the  cutting  edge  of 
preventive,  risk-reduction  work 
but  lack  the  resources  to 
systematically  assess 

and  disseminate  their  work. 


A  typical  full-service  community- 
based  family-centered  organization 
delivers  a  variety  of  services  to  its 
clients.  These  services  can  be 
described  in  two  separate,  but 
overlapping  groupings,  Home  based  and 
Community  based,  as  outlined  in  the 
next  two  Tables. 

Typically,  many  of  these  programs 
are  funded  by  different  funding 
streams,  with  each  funding  agency 
requiring  different  reporting.  Some  of 
these  programs  request  only  headcounts 
of   unduplicated   services   provided 


1 .  Community  health  nursing 

2.  Social  work 

3 .  Case  management 

4.  Therapeutic  counseling  of 
families  affected  by  HIV/AIDS 

5 .  Pastoral  care 

6 .  Home  health  aide 

7.  Early  childhood  specialist 

8.  Parent/family  aide  services 

9.  Educational  liaison  to  school 


TABLE  5B  -  COMMUNITY  BASED  SERVICES 


1.  Crisis  walk-in  service 

2 .  HIV  education/outreach 

3 .  Legal  services 

4.  Mothers'  group  teaching 
parenting  and  other  life  skills 

5.  Therapeutic  playroom  for  high- 
risk  newborns  and  infants 

6 .  Drug  &  substance  abuse 
counseling  groups 

7.  Children  and  teen  groups 

8.  Support  for  HIV  affected 
individuals  and  families 

9.  Liaison  nurse  linking  a  local 
community  hospital  and  the 
agency. 


during  a  time  period.  A  preventive 
program  serving  high-risk  women  might 
be  required  to  describe  the 
participation  of  these  women  in  various 
programs.  By  integrating  these  various 
data-sets  one  is  able  to  get  a  rich 
profile  of  the  population  (e.g  mothers 
and  their  newborns,  individuals  and 
families)  served  by  the  agency,  their 
needs,  services  provided  and  aspects  of 
their  health  and  socio-economic  status. 
Furthermore,  this  information  provides 
longitudinal  data.  Usually,  little  is 
known  about  these  individuals  until 
they  enter  the  medical  and  social 
service  system  for  emergent  and 
reimbursable  care. 

In  these  agencies,  even  though 
their  primary  identity  is  as  a  "Family 
Health  Service",  the  range  of  services 
utilized  goes  far  beyond  direct  health 
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needs.  For  example,  Table  6  shows  the 
distribution  of  initial  presenting 
problems  at  one  of  these  agencies  over 
a  30  month  period. 


TABLE  6 


PRESENTING  PROBLEM 


FAMILY  HEALTH  AGENCY 
6,337  VISITS  (30  MONTH  PERIOD) 


1. 

Entitlements, 

public  ass' t . 

48 

8% 

2. 

Housing 

30 

3% 

3  . 

Emergency  food 

8 

6% 

4  . 

Family  problems 

4 

0% 

5. 

Health  related 

issues 

1 

9% 

6. 

Job  training,  education 

1 

9% 

7. 

Other  (legal, 

immigration . . . ) 

1 

4% 

Note  the  relatively  minor  position 
occupied  by  'Health  Related  Issues'  at 
this  Family  Health  Service  agency  among 
the  multiple  problems  its  clients  face 
on  a  daily  basis.  Most  of  the  services 
being  sought  and  delivered  are  not 
insurance  reimbursable. 

In  order  to  properly  describe  the 
services  delivered,  and  to  begin  to 
derive  a  profile  of  service  needs  for 
this  special  population,  it  was 
necessary  to  capture  far  more 
information  in  retrievable  form  than  is 
usually  utilized  for  analysis  of 
service  utilization.  Data  are 
collected  at  each  visit  to  the  agency 
or  to  the  client's  home;  the  data 
collection  instruments  and  methods  of 
recording  data  are  flexible  enough  to 
accommodate  to  many  types  of  data.  In 
addition,  at  every  encounter,  one  or 
more  outcomes  is  recorded  as  well  as 
what  type  of  follow-up  is  needed  and 
what  referrals,  if  any,  are  made.  This 
applies  to  visits  scheduled  for  a 
specific  purpose  and  to  informal  'drop- 
in'  visits  when  the  client  comes  to  ask 
for  or  about  something.  Finally  each 
visit  measures  the  overall  risk  status 
of  the  client,  with  a  numerical  scale 
established  and  recorded,  to  serve  as  a 
flag  for  needed  follow-up. 

Most  programs  are  resistant  to  the 
introduction  of  data  collection  because 
of  staff's  experience  that  data 
collection  adds  work  with  no  payback 
for  their  efforts.  Program  specific 
data  are  often  collected  only  to  be 
shipped  off  in  reports  about  which 
little  feedback  is  provided  to  staff. 
With  this  program,  staff  soon  begin  to 
recognize  that  an  integrated  data- 
system  minimizes  the  data  that  need  to 
be  collected  and  helps  them  to  better 
understand  the  work  that  they  are 
doing.  Initially  administration  and 
staff  most  readily  understand  the  value 
of  the  system  for  generating  mandated 
reports,  internal  and  agencywide  lists 
and  program  and  productivity  analysis. 
Over  time,  they  value  its  ability  to 


provide  them  with  the  tools  with  which 
to  do  on-going  assessment  and  program 
revisions . 

In  establishing  and  implementing 
the  MIS  system  at  all  of  these 
agencies,  we  have  followed  a  schedule 
of  actions  and  activities  which  is 
outlined  in  the  following  table. 


TABLE  7 


IMPLEMENTATION  OF  MIS  SYSTEM 


1.  Agency  assessment  and  planning 

2.  Initial  system  specification 

3 .  System  design 

4 .  Analysis  of  the  flow  of 
paperwork  and  forms  revisions 

5.  Implementation 

6 .  Reporting 

7.  On-going  assessment/revisions  of 
program 

8.  System  enhancement  -  extensions- 
after  the  dust  has  settled 

9.  Evaluation,  research  and 
development. 


This  overall  approach  has  been 
used  at  each  type  of  site,  with  minor 
variations.  There  is  no  set  timetable 
for  any  section  of  the  implementation 
plan.  Much  depends  on  how  ready  the 
agency  is  to  commit  the  appropriate 
level  of  resources  in  time,  people  and 
dollars  to  the  project.  It  is 
instructive  to  examine  elements  of  the 
initial  step,  Agency  Assessment  and 
Planning,  as  shown  in  Table  8. 

TABLE  8  -  AGENCY  ASSESSMENT /PLANNING 


1.  Mandated  reporting 

2 .  Other  reports  needed  for 
program  operation 

3a.  Data  currently  collected  -  new 
and  existing  clients 

3b.  Data  forms  in  use 

4.  Confidentiality  of  data  - 
mandated  or  desired  levels 

5.  Number  of  clients,  number  of 
visits,  by  program; 

6 .  Source  of  referrals  - 
accompanying  paperwork 

7 .  Survey  of  agency-wide 
distribution  of  information- - 
what  is/is  not  shared  with 
other  programs  -   duplication 

of  efforts. 


400 


Note  that  we  start  by  examining  in 
great  detail  the  Reporting  function  of 
the  system.  We  feel  that  it  is 
mandatory  to  start  at  what  is  often 
considered  the  'back  end'  of  a  system. 
After  all,  the  object  of  installing  a 
system  is  not  to  collect  data,  but  to 
utilize  the  data  for  managing,  planning 
and  analyzing. 

One  of  the  most  critical  decisions 
which  has  to  be  made  at  the  outset  is 
to  make  certain  that  the  data  which  are 
collected  contain  the  appropriate 
'units  of  measure'  to  enable  proper 
analysis  of  the  data  at  hand  and,  of 
greater  importance,  to  allow  linking  of 
the  datasets  with  other  types  of  data 
and  datasets.  To  do  this  one  has  to  be 
aware  of  how  other  types  of  data  which 
one  will  want  to  use  to  augment  the 
local  data  are  being  collected  and 
arranged . 

For  example,  if  you  are  going  to 
utilize  household  income  data  as 
collected  by  the  Census,  then  you  have 
to  be  certain  to  incorporate  a  field  in 
your  client  address  database  to 
identify  the  Census  Tract  corresponding 
to  each  address.  As  another  example, 
epidemiologic  data  available  from  the 
New  York  City  Department  of  Health  are 
arranged  by  HSA  area;  in  order  to  be 
able  to  link  your  data  to  the  NYC  data, 
each  client  address  record  should  also 
include  the  HSA  area.  With  respect  to 
other  units  of  measure  such  as 
ethnicity,  country  of  origin,  or 
country  of  birth,  you  may  be  collecting 
information  which  is  much  more  detailed 
than  the  standard  5  ethnic  groups  used 
for  much  reporting,  but  you  have  to 
incorporate  appropriate  hooks  in  your 
data  (this  is  very  easy  to  do  using 
small  look-up  databases)  which  will 
allow  you  to  summarize  your  data  into 
the  standard  5  groups. 

The  system  utilizes  Alpha  Four 
(Alpha  Software)  for  data  management 
and  R&R  Report  Writer  (Concentric 
Software)  for  all  reporting.  Both  of 
these  very  sophisticated  application 
programs  are  based  on  the  dBASE  file 
structure.  We  chose  these  application 
programs  because  of  their  nearly 
limitless  flexibility  -  the  system  has 
to  be  able  to  quickly  accommodate 
changes  in  database  fields,  display 
screens,  and  reporting  requirements.  A 
closed  system  which  cannot  be  custom 
modified  to  suit  needs  of  each  agency 
simply  will  not  be  useful  for  any 
agency.  It  will  serve  only  to  record 
the  same  minimal  core  set  of  data  from 
each  agency,  but  in  general  will  not 
give  anything  back  to  the  agency. 

At  a  family-based  home  health 
agency  and  for  many  other  types  of 
service  agencies,  services  are 
generally  delivered  to  individuals 
with : 


(a)  many  individuals  served  by  more 
than  one  program; 

(b)  many  households  containing  more 
than  one  individual  receiving  services. 

The  file  structure  we  utilize 
includes  separate  databases  for 
HOUSEHOLD  demographics  and  for 
INDIVIDUAL  client  demographics.  A 
unique  code  for  each  household  and  for 
each  client  links  the  databases  and 
enables  tracking  of  all  services 
provided  to  each  client  in  all 
programs,  and  further,  allows  tracking 
of  the  totality  of  services  provided  to 
each  household  unit . 

For  each  agency  program,  at  least 
two  program-specific  databases  are 
established;  an  INTAKE  database  to 
capture  the  data  uniquely  required  by 
that  program,  and  a  VISIT/LOG  database 
to  record  the  daily  (or  weekly) 
activities  of  each  client  within  the 
program.  These  databases  differ  from 
program  to  program  since  the  tracking 
needs  of  an  Education  program  are  quite 
different  from  those  of  a  Nursing 
program. 

When  we  link  the  data  for  program 
VISITS  to  the  INTAKE,  INDIVIDUAL  and 
HOUSEHOLD  databases  into  a  SET,  we 
utilize  the  power  of  a  relational 
database  structure  and  can  examine  and 
analyze  data  using  the  very  powerful 
selection,  sorting,  and  subtotaling 
features  of  the  Report  Writer. 

In  Tables  10  and  11  we  present 
examples  of  data  extracted  from  the 
system  for  several  different  agencies. 

TABLE  10  -  DEMOGRAPHIC  DATA 
FAMILY  HEALTH  SERVICES 


AGENCY  A    AGENCY  B 
3.5  YRS      18  MOS 

Families  3,065  1,855 
Individuals  7,701  5,200 
Visits  Walk-in  8,567      4,100 

Outreach  3,400 

Nursing  4,800 

Ryan/White  2,200 

Mat/Child  1,100 


ETHNIC  GROUPS  >  200 
Puerto  Rican 
Afro  American 
Mexican 

Dominican  Republic 
Other  Central  American 
*  *  *  + 

HOUSEHOLD  SIZE 

1  -  30% 

2  -  25%  (high  %  single  parents) 

3  -  18% 

4  -  11% 
5-5% 
>5-  11% 
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1.  Because  of  the  substantial  size 
of  the  various  Hispanic  subgroups,  both 
agencies  have  been  able  to  target  their 
hiring,  their  outreach  programs  and 
their  services  very  specifically.  The 
very  detailed  visit  records,  for  some 
clients  spanning  3  years,  allow  us  to 
see  how  identified  problems  are 
followed  up  and  resolved  and  allow  for 
modification  of  assessment  criteria. 

2.  The  maternal  child  project, 
which  actually  encompasses  several 
different  agency  programs,  allows 
detailed  prenatal  assessments  and 
interventions  of  high  risk  mothers  to 
be  to  be  linked  to  follow-up  newborn 
interventions  and  to  identify  new 
clusters  of  prenatal  risk  factors. 

3  .  The  data  on  the  Ryan-White 
group  of  280  individuals  belonging  to 
about  200  different  households  allows 

detailed  analysis  of  the  impact  of  HIV 
and  AIDS  on  family  units  at  various 
stages  in  the  disease  cycle. 

Table  11  presents  data  for  two 
different  types  of  programs,  a  Needle 
Exchange  and  the  Mount  Sinai  AIDS 
Center : 


TABLE  11  -  DEMOGRAPHIC 

DATA 

NEEDLE 

AIDS 

EXCHANGE 

CENTER 

INDIVIDUALS 

>   12,000 

3 

,628 

VISITS 

>   35,000 

24 

,000 

HOSPITALIZATIONS 

N/A 

4 

844 

NEEDLES  DISTRIB 

>  300,000 

N/A 

**** 

MALE 

76% 

64% 

FEMALE 

24% 
**** 

36% 

HISPANIC 

51% 

49% 

AFRICAN  AMERICAN 

r   34% 

36% 

CAUCASIAN 

14% 

13% 

OTHER/UNKNOWN 

1% 

2% 

1.  It  is  interesting  to  note  that 
the  ethnic  distribution  of  both 
populations  is  nearly  identical,  but 
females  are  under-represented  at  the 
Needle  Exchange.  On  the  basis  of  an 
analysis  of  the  minimal  datasets 
collected  at  the  exchange  which 
examined  utilization  and  retention 
rates  of  clients,  we  were  able  to 
obtain  a  grant  for  a  TB  DOPT  program 
sited  at  the  Needle  Exchange  field 
location,  which  has  served  as  a 
pipeline  to  medical,  social  services 
and  drug  treatment  services. 

2  .  The  AIDS  Center  databases  cover 
both  clinic  visits  and 
hospitalizations,  thus  allowing  us  to 
follow  the  course  of  the  patients  and 
the  disease  in  a  greater  level  of 
detail  than  if  one  were  to  look  only  at 
inpatient  or  outpatient  data   -   for 


example  comparing  those  whose  first 
contact  was  through  the  Clinic  with 
those  who  were  first  hospitalized. 

As   each   program   has   become 

operational,  the  CBO  has  experienced 
the  standard  (and  anticipated)  reaction 
to  the  availability  of  computerized 
reports,  namely,  a  desire  to  modify 
aspects  of  their  operating  procedures 
and  data  collection  in  response  to  a 
recognition  that  they  could  now  ask 
questions  of  the  data  which  were 
virtually  impossible  when  using  paper 
records  only.  As  a  result,  each 
program's  individualized  databases  and 
associated  reports  have  been  revised  an 
average  of  twice  a  year. 

From  these  examples  we  see  that 
each  agency  is  increasingly  able  to 
utilize  agency-wide  reporting  to  plan 
for  future  needs.  (The  needle  exchange 
program  was  able  to  seek  and  obtain 
Ryan  White  funding  to  develop  a  TB 
screening  and  preventive  therapy 
program;  the  Mental  Health  Program  has 
been  able  to  obtain  additional  funding; 
one  of  the  two  Family  Health  Services 
agencies  is  linking  Ryan  White  Title  I 
and  Title  IV  program  data,  the  AIDS 
Center  program  has  been  able  to  modify 
existing  forms  and  data-sets  to  capture 
new  information  required  by  outside 
auditors) . 

CONCLUSIONS 

Large  and  small  agencies  have  been 
grappling  with  the  information 
revolution  that  is  allowing  and  forcing 
the  collection  and  organization  of 
masses  of  data  and  the  detailed 
reporting  of  program  activities  and 
resources .  The  advent  of  PCs  and 
powerful  data-base  and  reporting 
software,  if  supported  by  sustained 
technical  assistance  from  social 
scientists  and  system  analysts,  allows 
the  smallest  agencies  to  organize, 
analyze  and  modulate  and  expand 
programs  to  meet  internal  and  external 
forces.  The  technology  is  available 
which  allows  small  agencies  to  manage 
and  analyze  complex  data.  However,  to 
facilitate  the  adoption  of  this 
approach  requires  the  development  of 
collaboration  among  agency  program  and 
administration,  and  a  consultation  team 
that  provides  sustained  program,  staff, 
resource  and  data  development  and 
evaluation  support . 

This  paper  offers  a  feasible 
approach  for  harnessing  and  the 
utilizing  these  strategic  grassroots 
sites  involved  in  upstream  work  with 
special  populations  as  sites  in  which 
to  collect  and  aggregate  rich  local 
geographic  information  to  monitor  and 
assess  and  address  the  emergent  needs 
in  vulnerable  communities  and  regional 
areas . 
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HOSPITAL  REPORTING  PRACTICES  AND  THEIR  IMPACT  ON 
TEXAS  BIRTH  CERTIFICATE  DATA  QUALITY 

Mary  K.  Ethen,  Texas  Department  of  Health 
Beatrice  J.  Selwyn  and  Stephen  B.  Borders 


In  1989,  following  national  recommenda- 
tions, the  Texas  birth  certificate  was  extensive- 
ly revised  (1).  The  certificate  now  includes  a 
series  of  check  box  items  to  collect  information 
on  more  than  50  specific  medical  conditions  and 
procedures  related  to  pregnancy,  childbirth  and 
the  health  of  mothers  and  newborns  (see 
Appendix)  .  This  was  intended  to  provide  data  for 
public  health  monitoring,  community  health  needs 
assessment,  epidemiologic  research,  and  program 
planning  and  evaluation. 

This  paper  presents  the  results  of  two 
surveys  related  to  birth  certificate  data  quali- 
ty. The  first  survey  evaluates  the  completeness 
and  accuracy  of  birth  certificate  data,  and  the 
second  survey  describes  hospital  practices  for 
collecting  birth  certificate  information  which 
affect  data  quality.  Based  on  these  studies, 
recommendations  are  presented  for  improving  the 
quality  of  health-related  information  collected 
via  birth  certificates. 

VALIDATION  OF  BIRTH  CERTIFICATE  DATA 

The  first  survey  arose  as  a  preliminary 
investigation  for  a  larger  project  that  proposed 
to  use  the  new  health  and  medical  information  on 
birth  certificates  to  identify  risk  factors  for 
infant  mortality  and  other  adverse  pregnancy 
outcomes  in  a  metropolitan  area  of  Texas.  Howev- 
er, before  using  the  new  data  for  this  purpose, 
the  investigators  decided  to  first  assess  the 
validity  of  the  health  and  medical  information 
reported  on  birth  certificates  in  comparison  to 
hospital  medical  records. 

Methods 

Sample  Selection   and  Data  Collection  Procedures 

We  restricted  the  study  to  births  that 
occurred  during  1991  in  the  six  largest  hospitals 
in  the  county  of  interest.  These  six  hospitals 
accounted  for  more  than  half  of  all  births  in  the 
county.  We  selected  a  systematic  probability 
sample  of  443  total  live  births,  one  out  of  every 
74  deliveries  in  the  each  of  the  six  target 
hospitals.  In  this  way,  the  number  of  births 
selected  from  each  hospital  was  proportional  to 
the  total  number  of  live  births  in  that  hospital. 

For  each  birth  selected,  we  requested  the 
newborn's  and  the  mother's  hospital  medical 
records.  Without  knowledge  of  what  had  been 
reported  on  the  birth  certificate,  trained  ab- 
stracters carefully  reviewed  both  the  mother's 
and  the  infant's  medical  records  for  421  (95 
percent)  of  the  births  selected.  Abstracters 
entered  data  from  the  medical  records  directly 
into  laptop  computers  while  at  the  hospital, 
using  computerized  data  collection  instruments 
designed  in  Epi  Info  version  5.01B  (2).  Range 
and  logic  checks  were  run  at  the  time  of  data 
entry,  which  allowed  us  to  detect  and  correct 
many  potential  errors  and  logical  inconsistencies 
while  we  had  the  medical  records  in  hand.  The 
data  abstracted  from  the  medical  records  were 
then  matched  to  computerized  birth  certificate 
data  for  the  same  birth,  and  the  two  sources  of 
information  were  compared  for  similarity. 

Measures  of  Data   Validity 

We  used  three  measures  to  assess  the  valid- 
ity of  birth  certificate  information  in  compari- 
son to  medical  records  data  (Table  1) .  The  first 
measure,  which  was  calculated  for  each  item  of 
information   collected,   is   percent   agreement. 


Percent  agreement  is  the  percentage  of  births 
reviewed  for  which  the  information  reported  on 
the  birth  certificate  agreed  with  what  we  found 
in  the  medical  records . 

For  the  medical  conditions  and  procedures 
reported  on  the  birth  certificate  using  check 
boxes,  percent  agreement  consists  of  two  types  of 
agreement:  births  for  which  the  birth  certifi- 
cate and  medical  records  both  indicate  that  a 
given  condition  or  procedure  existed,  plus  births 
for  which  the  two  sources  indicate  that  the 
condition  or  procedure  did  not  exist.  Because 
most  births  are  normal  and  without  complications, 
there  are  many  births  for  which  the  birth  certif- 
icate and  the  medical  records  will  agree  that 
given  conditions  or  procedures  did  not  exist. 
This  results  in  high  levels  of  agreement  between 
birth  certificates  and  medical  records  for  most 
medical  conditions,  simply  because  most  of  the 
reportable  conditions  are  rare. 

For  this  reason  two  additional  measures  of 
data  validity,  sensitivity  and  predictive  value 
positive,  were  used  to  evaluate  the  medical 
information  reported  on  birth  certificates  (Table 
1) .  Sensitivity  is  the  fraction  of  births  with  a 
given  condition  according  to  the  medical  records 
which  had  that  condition  reported  on  the  birth 
certificate.  Sensitivity  indicates  completeness 
of  case  identification  on  the  birth  certificate. 
Predictive  value  positive  is  the  fraction  of 
births  with  a  given  condition  reported  on  the 
birth  certificate  which  had  that  condition 
according  to  the  medical  records.  Predictive 
value  positive  indicates  accuracy  of  case 
identification  on  the  birth  certificate. 


Table  1 
Measures  of  birth  certificate  data  validity 


1.   Percent  Agreement: 

number  of  births  for  which  birth 
certificate  &  medical  records  agree 

number  of  births  reviewed 


x  100 


Indicates  similarity   of  birth  certificate 
and  medical  records  information 


2.   Sensitivity: 

number  of  births  with  condition  reported  in 
medical  records  and  on  birth  certificate 

number  of  births  with  condition  reported  in 
medical  records 

•   Indicates  completeness   of  case 

identification  on  the  birth  certificate 


i.      Predictive  value  positive: 

number  of  births  with  condition  reported  in 
medical  records  and  on  birth  certificate 

number  of  births  with  condition  reported  on 
birth  certificate 

>   Indicates  accuracy   of  case  identification 
on  the  birth  certificate 
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Results 

We  found  97  to  100  percent  agreement  be- 
tween the  birth  certificate  and  corresponding 
medical  records  for  basic  information  such  as 
date  of  birth,  sex,  birth  weight  and  mother's 
date  of  birth  (Table  2).  Method  of  delivery  and 
last  menstrual  period  date  were  less  accurately 
reported  on  birth  certificates:  each  showed  82 
percent  agreement  with  medical  records  data. 
Only  53  percent  of  the  birth  certificates  re- 
viewed agreed  with  the  medical  records  for  the 
month  of  pregnancy  prenatal  care  began. 

Most  of  the  medical  conditions  and 
procedures  reported  on  the  certificate  via  check 
boxes  fell  in  the  range  of  80  to  90  percent 
agreement.  This  agreement  consisted  almost 
entirely  of  normal,  uncomplicated  pregnancies  for 
which  the  birth  certificate  and  medical  records 
both  indicated  that  various  conditions  did  not 
exist . 


Table  2 

Percentage  of  births  with  agreement  between 

birth  certificate  and  medical  records  data, 

Texas,  1991 


Data  item 


%  agreement 


Sex  100 

Date  of  birth  99 
Birth  weight 

(within  +1  oz.  or  +28  g.)  97 

Mother's  date  of  birth  97 

Method  of  deliveryt  82 
Last  menstrual  period  date 

(month  and  year  only)  82 
Month  prenatal  care  began 

(within  ±1  month)  53 

t  vaginal,  vaginal  after  previous  C-section, 

primary  C-section  or  repeat  C-section 


To  assess  how  often  specific  conditions  and 
procedures  documented  in  the  medical  records  were 
reported  on  the  birth  certificate,  we  computed 
sensitivity  (see  Table  1  for  definition) .  For 
example,  we  found  33  mothers  who  had  diabetes 
according  to  their  medical  records;  only  15  of 
them  had  diabetes  reported  on  the  birth 
certificate  (Table  3) .  We  found  26  mothers  with 
pregnancy-associated  hypertension  according  to 
their  medical  records;  only  7  of  these  were 
reported  on  the  birth  certificate.  Only  5  of  60 
cases  of  a  sexually  transmitted  disease  during 
pregnancy  were  reported  on  the  birth  certificate. 
Sensitivity  was  zero  percent  for  24  reportable 
conditions  and  procedures,  including  previous 
preterm  or  small-for-gestational-age  infant, 
tocolysis,  and  dysfunctional  labor. 

To  evaluate  how  often  conditions  and 
procedures  reported  on  the  birth  certificate  were 
verified  by  the  medical  records,  we  computed 
predictive  value  positive  (for  definition  see 
Table  1) .  There  were  29  birth  certificates  with 
induction  of  labor  reported,  but  only  15  of  these 
were  induced  according  to  the  medical  records 
(Table  3)  .  Only  3  of  9  reports  of  a  fever  during 
labor  were  documented  in  the  medical  records . 
Predictive  value  positive  was  very  high  for  fetal 
monitoring:  188  of  196  cases  reported  on  the 
birth  certificate  were  confirmed  by  the  medical 
records.  However,  the  sensitivity  for  fetal 
monitoring  was  low.  According  to  the  medical 
records,  410  women  received  fetal  monitoring; 
only  188  of  these  were  reported  on  the  birth 
certificate . 

Predictive  value  positive  was  100  percent 
for  6  conditions,  including  diabetes  and  pregnan- 
cy-associated hypertension,  meaning  that  all  of 


the  cases  reported  on  the  birth  certificate  were 
confirmed  by  the  medical  records.  At  the  other 
extreme,  predictive  value  positive  was  zero 
percent  for  11  conditions,  meaning  that  none  of 
the  cases  reported  on  the  birth  certificate  were 
supported  by  the  medical  records.  However,  all 
11  conditions  with  zero  percent  predictive  value 
positive  were  reported  on  the  birth  certificates 
in  our  sample  no  more  than  4  times. 

These  findings  indicate  that,  for  most 
conditions  and  procedures,  only  a  portion  of  the 
cases  documented  in  the  medical  records  were 
reported  on  the  birth  certificate;  for  many 
conditions,  none  of  the  cases  found  in  the 
medical  records  were  reported  on  the  certificate. 
Furthermore,  conditions  reported  on  the  birth 
certificate  were  not  always  corroborated  by  the 
medical  records. 

BIRTH  CERTIFICATE  REPORTING  PRACTICES  SURVEY 

We  recognized  that  there  was  a  problem 
getting  health-related  information  from  the 
medical  records  onto  birth  certificates,  but  we 
didn't  know  exactly  where  the  breakdown  was 
occurring.  In  order  to  gather  information  on 
birth  certificate  reporting  practices  that  affect 
data  quality,  a  telephone  survey  of  personnel 
responsible  for  preparing  birth  certificates  in 
hospitals  throughout  Texas  was  conducted  in  1994. 

Methods 

Live  births  were  reported  in  328  Texas 
hospitals  in  1993.  Our  strategy  was  to  survey 
more  of  the  hospitals  that  prepare  a  large  number 
of  birth  certificates  and  fewer  hospitals  that 
prepare  a  small  number  of  birth  certificates.  We 
excluded  from  the  survey  hospitals  with  50  or 
fewer  births  per  year  (n=63  hospitals),  because 
they  averaged  less  than  1  birth  per  week  and 
contributed  only  0.2  percent  of  all  1993  Texas 
occurrence  births.  The  remaining  265  hospitals 
were  divided  into  four  groups  (strata)  based  on 
number  of  births  in  1993,  with  cut-points  between 
strata  chosen  to  most  closely  satisfy  the  follow- 
ing guideline:  2  times  the  division  point  be- 
tween stratum  x  and  stratum  (x+1)  should  equal 
the  mean  number  of  births  in  stratum  x  plus  the 
mean  number  of  births  in  stratum  (x+1)  (3,pg.96). 

From  each  group,  we  selected  a  number  of 
hospitals  proportional  to  the  percentage  of 
births  that  occurred  in  all  hospitals  belonging 
to  that  group  (Table  4) .  For  example,  we  found 
that  10  percent  of  all  births  occurred  in  hospi- 
tals with  51  to  600  births  per  year,  so  we  made 
sure  that  10  percent  of  our  sample  consisted  of 
hospitals  of  that  size.  In  this  way,  our  sample 
was  weighted  to  represent  the  distribution  of 
Texas  births  according  to  hospital  size.  Our 
total  sample  size  was  limited  to  71  hospitals  by 
the  fact  that  44  percent  of  all  Texas  in-hospital 
births  occurred  in  the  31  largest  hospitals  (31 
equals  44  percent  of  71) .  The  31  largest  hospi- 
tals were  all  selected  for  the  survey,  and  a 
systematic  sample  of  hospitals  was  selected  from 
within  each  of  the  other  three  strata. 

We  telephoned  each  selected  hospital, 
invited  the  staff  member  responsible  for  prepar- 
ing birth  certificates  to  participate,  and  sched- 
uled a  convenient  time  to  call  back  and  conduct 
the  interview.  Hospital  personnel  were  asked 
questions  about  the  methods  they  use  to  collect 
and  report  birth  certificate  information,  and 
about  their  education,  experience  and  training. 
Interviews  lasted  approximately  3  0  minutes  and 
were  completed  with  69  of  the  71  hospitals  se- 
lected, for  a  97  percent  response  rate.  The 
hospitals  surveyed  were  located  throughout  the 
state  and  contributed  57  percent  of  all  1993 
Texas  births.  Public  hospitals,  private  not-for- 
profit  hospitals,  private  for-profit  hospitals 
and  military  hospitals  were  all  represented  in 
the  survey . 
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Table  3 
Sensitivityt  and  predictive  value  positivet  for  selected  health  and  medical  information 
reported  on  the  birth  certificate  using  check  boxes,  Texas,  1991 


Data  item 


Sensitivity 


Predictive 
value  positive 


Data  item 


Sensitivity 


Predictive 
value  positive 


Maternal  Medical   Risk  Factors 


Maternal  anemia 

2/50 

Cardiac  disease 

0/9 

Lung  disease 

0/15 

Diabetes 

15/33 

Hydramnios/ 

oligohydramnios 

0/25 

Hemoglobinopathy 

0/2 

Chronic  hyper- 

tension 

2/8 

Pregnancy-associated 

hypertension 

7/26 

Eclampsia 

Previous  infant 

400  0+  grams 

0/21 

Previous  preterm  or 

small-for-gestatione 

l1- 

age  infant 

0/31 

Renal  disease 

0/12 

Rh  sensitization 

0/3 

Preterm  rupture  of 

membranes  (<37  wks) 

1/42 

Sexually  transmitted 

disease 

5/60 

Obstetric  Procedures 
Amniocentesis  10/42 
Fetal  monitoring  188/410 
Induction  of  labor  15/71 
Stimulation  of  labor  5/206 
Tocolysis  0/22 

Ultrasound  138/305 


2/7 
15/15 

2/2 

7/7 
0/1 

0/1 

0/2 
0/4 
1/3 
5/6 


10/15 

188/196 

15/29 

5/8 

0/2 

138/170 


Complications   of  Labor  and  Delivery 


Febrile 

Meconium,  moderate 

or  heavy 
Abruptio  placenta 
Placenta  previa 
Other  excessive 

bleeding 
Seizures  during  labor 
Precipitous  labor 
Prolonged  labor 
Dysfunctional  labor 
Breech/malpresentation  8/33 
Failure  to  progress  in 

labor  (cephalopelvic 

disproportion)        7/12 
Cord  prolapse  0/1 


3/22 

11/39 
1/6 
1/4 

0/3 

1/10 
0/15 
0/52 


3/9 

11/28 
1/1 
1/1 

0/1 

1/4 

0/1 
8/11 


7/11 
0/1 


Abnormal    Conditions   of   the  Newborn 
Fetal  alcohol  syndrome 
Hyaline  membrane 

disease /respiratory 


distress  syndrome 

0/11 

0/1 

Meconium  aspiration 

syndrome 

1/2 

1/12 

Assisted  ventilation 

<3  0  minutes 

3/22 

3/19 

30+  minutes 

0/11 

0/3 

Seizures 

0/1 

Congenital  Anomalies   of   the  Newborn 
Anencephalus 
Spina  bifida/meningocele 
Hydrocephalus 

Microcephalus  0/1 

Other  central  nervous 

system  anomalies 
Heart  malformations     0/4 
Other  circulatory/ 

respiratory  anomalies  0/8 
Rectal  atresia/stenosis 
Tracheo-esophageal  fistula 

/esophageal  atresia 
Omphalocele 

/gastroschisis        0/9 
Other  gastrointestinal 

anomalies  1/9  1/1 

Malformed  genitalia     0/8 
Renal  agenesis 
Other  urogenital 

anomalies 
Cleft  lip/palate        0/1 
Polydactyly/syndactyly   0/2 
Limb  reduction(s)  0/1 

Club  foot 

Diaphragmatic  hernia 
Other  musculoskeletal/ 

integumental 

anomalies  0/4 

Down ' s  syndrome 
Other  chromosomal 

anomalies 


t  See  Table  1  for  definitions.   A  blank  for  sensitivity  means  there  were  no  occurrences  in  the  medical 
records.   A  blank  for  predictive  value  positive  means  there  were  no  occurrences  on  the  birth  certificate. 


Results 

The  hospital  employee  responsible  for 
preparing  birth  certificates  is  usually  a  medical 
records  clerk  or  a  ward  clerk  in  the  maternity 
department.  Birth  certificate  clerks  have  a 
median  of  13  years  of  education  (Table  5) . 
Thirty-eight  percent  have  no  more  than  a  high 
school  education;  10  percent  have  completed 
college . 

Nearly  one-quarter  of  the  personnel  sur- 
veyed have  less  than  one  year's  experience  pre- 
paring birth  certificates,  which  indicates  the 
rate  of  turnover  in  these  positions  (Table  5) . 
The  median  longevity  is  three  years. 

With  the  turnover  in  these  positions,  it's 
not  surprising  that  the  majority  of  birth  certif- 


Table  4 
Hospital  stratification  and  sample  selection 


Hospital 

size 

according 

Total 
hospitals 

%  of 

total 

1993 

hospital 

births 

Hospitals 
selected 

to  births 
in  1993 

# 

%  of 
sample 

51-600 
601-1500 
1501-2900 
2901  + 

118 
82 
34 
31 

10% 
24% 
22% 
44% 

7 
17 
16 
31 

10% 
24% 
22% 
44% 

Total 


265 


100% 


71 


100% 
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Table  5 

Education  and  experience  of  hospital  clerks  who 

prepare  birth  certificates,  Texas,  1994 


%  of  hospitals 
surveyed 


Years  of  education  completed 

11  to  12  38 

13  to  15  51 

16  or  more  10 

Median:  13  years 

Years  preparing  birth  certificates 

less  than  1  23 

1  to  3  28 

4  to  8  25 

9  or  more  25 
Median:  3  years 

Percentages  may  not  sum  to  100  due  to  rounding. 


Table  6 

Who  taught  hospital  clerks  to  prepare  birth 

certificates,  Texas,  1994 


%  of  hospitals 
surveyed 


Predecessor  66 

Supervisor  3  5 

Self-taught  34 

Vital  statistics  field  rep  19 

Local  registrar  18 

Vital  statistics  conference  13 

Employee  of  another  hospital  12 

Multiple  responses  allowed,  percentages  sum  to 
greater  than  100. 


icate  clerks  were  trained  by  their  predecessors 
or  their  supervisors,  or  taught  themselves  using 
written  instructions  from  the  Texas  Bureau  of 
Vital  Statistics  (Table  6)  .  Nineteen  percent  said 
they  were  taught  by  the  vital  statistics  field 
representative,  and  13  percent  said  they  learned 
at  the  annual  vital  statistics  conference. 

Since  clerical  personnel  are  responsible 
for  preparing  birth  certificates,  we  asked  survey 
respondents  whether  anyone  summarizes  the  health 
and  medical  information  they  need  for  the  certif- 
icate. We  categorized  hospitals  according  to  the 
most  highly  trained  staff  member  who  provides 
health  and  medical  information  for  birth  certifi- 
cates. We  found  that  a  physician  or  nurse  pro- 
vides the  medical  data  in  48  percent  of  hospitals 
surveyed,  a  medical  records  coder  supplies  this 
information  in  3  percent  of  hospitals,  and  in  49 
percent  of  hospitals,  the  birth  certificate  clerk 
collects  the  medical  information  by  checking  the 
patient's  records  (Table  7). 

While  physicians  and  nurses  summarize  the 
medical  data  in  48  percent  of  hospitals  surveyed, 
these  hospitals  contributed  only  40  percent  of 
all  births  that  occurred  in  the  surveyed  hospi- 
tals (Table  7).  Fifty-eight  percent  of  total 
births  occurred  in  facilities  where  a  clerk 
collects  the  health  and  medical  information 
required  for  the  certificate. 

We  asked  respondents  to  estimate  the  total 
number  of  hours  per  week  that  all  birth 
certificate  clerks  employed  by  the  hospital  spend 
working  on  any  aspect  of  birth  certificate 
preparation.  For  each  hospital  surveyed,  we 


Table  7 

Distribution  of  hospitals  and  births,  according 

to  most  highly  trained  staff  member  who  provides 

health  and  medical  information  for  birth 

certificates,  Texas,  1994 


Staff  member  who         %  of 
provides  health  and    hospitals 
medical  information     surveyed 


%  of  total 
births  in 
hospitals 
surveyed 


Clerk  49 

Physician  or  nurse         48 
Medical  records  coder       3 


58 

40 

3 


Total  100  100 

Percentages  may  not  sum  to  100  due  to  rounding. 


Table  8 

Time  needed  to  prepare  one  birth  certificate, 

Texas,  1994 


Hospital  size 
(births  per  year) 


Median  time  spent 
per  birth  certificate 


Small  (51-600)  69  minutes 

Medium  (601-1500)  56  minutes 

Large  (1501-2900)  52  minutes 

Very  large  (2900+)  40  minutes 

All  hospitals  surveyed  45  minutes 


related  clerk-hours  to  the  number  of  births  in 
the  hospital  to  calculate  the  average  time  needed 
to  complete  one  certificate. 

The  median  value  of  the  averages  from  each 
of  the  hospitals  surveyed  was  45  minutes  per 
birth  certificate  (Table  8) .  The  amount  of  time 
spent  per  certificate  was  inversely  proportional 
to  the  number  of  births  in  the  hospital:  smaller 
hospitals  devoted  more  time  to  birth 
certificates;  larger  hospitals  devoted  less  time. 

To  estimate  the  cost  of  preparing  birth 
certificates,  we  multiplied  the  median  time  of  45 
minutes  per  certificate  by  the  average  1993 
starting  wage  of  $5.68  per  hour  for  clerk-typists 
in  Texas  hospitals  (4,pg.29).  We  found  that  birth 
certificates  cost  hospitals  a  minimum  of  $4.26 
each  to  prepare  (Table  9)  .  This  is  a  minimum 
estimate,  and  does  not  include  other  costs  such 
as  fringe  benefits,  overhead  expenses,  supplies 
and  equipment,  or  the  time  spent  by  physicians, 
nurses  and  other  staff  members  who  provide  infor- 
mation for  birth  certificates. 

Multiplying  the  minimum  cost  of  $4.26  per 
certificate  by  318,642  in-hospital  births,  we 
found  that  birth  certificate  preparation  cost 
Texas  hospitals  more  than  1.35  million  dollars  in 
1993  (Table  9)  . 

SUMMARY  OF  FINDINGS  OF  THE  TWO  STUDIES 

To  recap,  we  learned  that  health  and  medi- 
cal information  reported  on  birth  certificates 
showed  fair  levels  of  agreement  with  medical 
records,  but  most  of  the  agreement  was  among 
normal,  uncomplicated  pregnancies.  When  medical 
conditions  or  procedures  existed  according  to  the 
hospital  records,  only  a  fraction  of  the  cases 
were  reported  on  the  birth  certificate.  Further, 
conditions  reported  on  birth  certificates  were 
not  always  confirmed  by  the  hospital  records. 
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Table  9 

Cost  of  birth  certificate  preparation, 

Texas,  1993 


45  minutes/certificate  x  $5.68/hour  = 
=>  $4.26  per  birth  certificate 

$4.26/certificate  x  318,642  in-hospital  births 
=$    over  $1.35  million  per  year 


less  time  to  complete,  and  medical  information 
would  only  be  required  on  selected  births,  hospi- 
tals should  be  able  to  devote  more  staff  time  or 
more  highly  trained  staff  members  to  collect  the 
medical  information.  We  might  also  consider 
doing  away  with  the  medical  information  on  birth 
certificates  altogether  and  collecting  it  through 
special  sample  surveys. 


Table  10 

Recommendations  for  improving  birth 

certificate  data  quality 


Health  and  medical  information  for  birth 
certificates  is  often  collected  by  clerical 
workers.  Wages  are  low,  there  is  regular  turn- 
over in  these  jobs,  and  most  clerks  were  taught 
to  complete  birth  certificates  by  the  person  who 
held  the  job  before  them. 

Birth  certificate  preparation  requires 
considerable  time  and  money.  In  spite  of  low 
wages,  birth  certificates  cost  Texas  hospitals 
over  1.35  million  dollars  per  year;  this  figure 
does  not  include  the  costs  borne  by  local 
registrars,  the  state  vital  statistics  office,  or 
the  National  Center  for  Health  Statistics. 
Substantial  amounts  of  time  and  money  are  being 
spent  collecting  data  that  are  of  poor  quality. 

These  findings  are  not  unique  to  Texas. 
Comparable  results  have  been  published  on  the 
validity  of  birth  certificate  data  in  Tennessee 
(5),  North  Carolina  (6),  and  Washington  State 
(7,8).  All  states  collect  a  similar  array  of 
information,  and  all  states  are  susceptible  to 
the  same  data  quality  problems. 

RECOMMENDATIONS  FOR  IMPROVING  BIRTH  CERTIFICATE 
DATA  QUALITY 

The  results  of  these  two  surveys  have  led 
us  to  several  ideas  for  improving  the  complete- 
ness and  accuracy  of  health-related  information 
on  birth  certificates  (Table  10) .  One  method  of 
improving  the  data  is  by  providing  more  training 
for  those  who  complete  certificates.  Because  of 
job  turnover,  training  programs  need  to  be  avail- 
able on  a  regular  basis.  Training  could  be  done 
via  satellite,  via  tutorials  built  into  computer 
programs  used  to  generate  birth  certificates,  or 
face-to-face.  Periodic  audits  of  a  sample  of 
birth  certificates  are  useful  to  identify  specif- 
ic problems  and  help  hospitals  develop  plans  to 
improve  data  quality. 

Clinicians  could  receive  training  regarding 
their  responsibilities  during  medical  or  nursing 
school,  during  internships  and  residencies,  or 
through  continuing  education  courses. 

The  birth  certificate  requests  information 
on  more  than  50  specific  medical  conditions  and 
procedures.  If  we  cannot  collect  this  informa- 
tion accurately  and  completely,  we  should  consid- 
er reducing  the  amount  of  information  collected. 
Floyd  Frost  and  colleagues  from  Washington  State 
noted  in  1984  that  "it  is  probably  more  useful  to 
have  relatively  complete  information  on  a  limited 
number  of  conditions  than  to  have  very  incomplete 
information  on  a  larger  number  of  conditions" 
(9,pg.506) . 

Perhaps  we  should  carefully  review  and 
revise  the  birth  certificate,  keeping  only  those 
items  most  important  from  a  public  health  point 
of  view,  then  concentrate  on  collecting  accurate 
information  on  a  smaller  set  of  conditions. 

Another  option  would  be  to  collect  exten- 
sive medical  information  on  a  sample  of  births. 
We  could  have  a  short  form  of  the  certificate, 
without  the  medical  information,  which  would  be 
used  routinely,  plus  a  long  form,  including  the 
health  and  medical  data,  to  be  used  on  a  sample 
of  births.   Because  the  short  form  would  take 


•  Provide  more  training  for  those  who  complete 
birth  certificates 

•  Reduce  the  amount  of  information  collected  via 
birth  certificates 

•  Increase  time  devoted  to  birth  certificates 
and  qualifications  of  staff  who  complete  birth 
certificates 

•  Automate  and  integrate  birth  certificate  data 
collection  into  computerized  patient  medical 
records 


A  third  idea  for  improving  the  quality  of 
birth  certificate  data  is  to  encourage  hospitals 
to  devote  more  staff  time  or  more  highly  trained 
staff  members  to  birth  certificate  preparation. 
There  will  probably  be  resistance  to  this,  as 
birth  certificate  preparation  is  not  a  revenue- 
generating  activity,  and  facilities  are  facing 
financial  challenges.  After  formal  training 
programs  are  established,  we  might  consider 
requiring  credentials  to  prepare  birth  certifi- 
cates. We  might  also  approach  accrediting  agen- 
cies to  tie  birth  certificate  accuracy  into 
facility  accreditation. 

Currently  Texas  and  other  states  have 
stand-alone  computer  programs  used  to  print  birth 
certificates  and  to  relay  the  data  to  the  state 
electronically.  The  next  step  is  to  integrate 
birth  certificate  preparation  into  computerized 
patient  records.  Programs  can  be  written  to 
automatically  extract  relevant  information  from 
computerized  medical  records,  and  place  that 
information  onto  the  birth  certificate.  The 
certificate  can  become  a  by-product  of  patient 
care,  rather  than  a  special  effort.  Automation 
will  reduce  the  time  needed  to  prepare 
certificates  and  increase  data  quality.  This 
will  be  a  challenge,  but  it  probably  has  the 
greatest  likelihood  for  payoffs,  because  it  will 
increase  data  quality  while  streamlining  hospital 
operations.  Now,  as  computerized  medical  record 
systems  are  being  developed,  is  the  critical  time 
for  vital  statistics  offices  to  work  with 
hospitals  and  software  vendors  to  integrate  birth 
certificates  into  computerized  medical  records 
systems . 

Forty-four  years  ago,  A.M.  Lilienfeld  and 
colleagues  from  the  New  York  State  Department  of 
Health  noted  that  "the  value  of  [routine  collec- 
tion of  morbidity  statistics]  depends  upon  the 
accuracy  of  the  reported  data  and  extent  of 
under-reporting"  (10, p. 192,.  The  current  data 
reporting  methods  are  not  providing  us  with 
accurate  and  complete  information.  State  vital 
statistics  offices,  departments  of  health,  and 
the  National  Center  for  Health  Statistics  must 
work  with  hospitals  to  improve  the  completeness 
and  accuracy  of  public  health  information  col- 
lected via  birth  certificates.  The  best  solu- 
tions are  those  which  offer  benefits  to  both  the 
data  providers  and  the  data  users.  Until  report- 
ing methods  and  data  quality  are  improved,  birth 
certificate  medical  information  must  be  used  with 
great  caution. 
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Appendix 
Health  and  medical  information  added  to  the  Texas  birth  certificate  in  1989 


34a.  MEDICAL  FACTORS  FOR  THIS  PREGNANCY 
(Check  all  that  apply) 

None  01 D 

Unknown 02D 

Anemia  (Hct.  <30/Hgb.<10) 03D 

Cardiac  disease 04  □ 

Acute  or  chronic  lung  disease 05D 

Diabetes  06D 

Hydramnios/Oligohydramnios   07D 

Hemoglobinopathy  08D 

Hypertension,  chronic 09D 

Hypertension,  pregnancy-associated 10  D 

Eclampsia 1 1  □ 

Incompetent  cervix  12  D 

Previous  infant  4000  +  grams 13D 

Previous  preterm  or  small-for-gestational-age 

infant 14D 

Renal  disease 15D 

Rh  sensitization 16D 

Preterm  rupture  of  membranes  (<  37  weeks) 17D 

STD  18D 

Other  19D 

(Specify) 

34b.  OTHER  FACTORS  FOR  THIS  PREGNANCY 
(Complete  all  items) 

Tobacco  use  during  pregnancy Yes  D  No  D 

Average  number  cigarettes  per  day 

Alcohol  use  during  pregnancy Yes  d  No  D 

Average  number  drinks  per  week 

Pre-pregnancy  weight lbs. 

Weight  gained  during  pregnancy lbs. 


35  OBSTETRIC  PROCEDURES 
(Check  all  that  apply) 

None  20D 

Amniocentesis 21 D 

Electronic  fetal  monitoring 22  D 

Induction  of  labor 23D 

Stimulation  of  labor 24D 

Tocolysis    25  D 

Ultrasound   26D 

Other  27  O 

(Specify) 


36.  EVENTS  OF  LABOR  AND/OR  DELIVERY 
(Check  all  that  apply) 

None   28D 

Febrile(>100°F.or38°C.) 29D 

Meconium,  moderate/heavy 30D 

Premature  rupture  of  membrane  (>  12  hours) 31 D 

Abruptio  placenta 32  D 

Placenta  previa 33D 

Other  excessive  bleeding 34  D 

Seizures  during  labor 35  □ 

Precipitous  labor  (<  3  hours) 36 D 

Prolonged  labor (> 20  hours) 37D 

Dysfunctional  labor 38  D 

Breech/Malpresentation    39D 

Failure  to  progress  in  labor 40D 

Cord  prolapse 41 D 

Other   42D 

(Specify) 


37.  METHOD  OF  DELIVERY 
(Check  all  that  apply) 

Vaginal   43D 

Vaginal  birth  after  previous  C-section 44  Q 

Primary  C-section 45D 

Repeat  C-section 46D 

Forceps , 47  D 

Vacuum ...-,. 48D 

38.  ABNORMAL  CONDITIONS  OF  THE  NEWBORN 
(Check  all  that  apply) 

None  49D 

Anemia(Hct.<39/Hgb,<13) 50D 

Fetal  alcohol  syndrome 51 D 

Hyaline  membrane  disease/RDS 52D 

Meconium  aspiration  syndrome 53D 

Assisted  ventilation  <30min 54D 

Assisted  ventilation  ^.30  min 55D 

Seizures    56D 

Other  57  D 

(Specify) 


39.  CONGENITAL  ANOMALIES  OF  CHILD 
(Check  all  that  apply) 

None  noted 58D 

Anencephalus   59D 

Spina  bifida/Meningocele 60D 

Hydrocephalus 61 D 

Microcephalus  62D 

Other  central  nervous  system  anomalies 
(Specify)    63D 

Heart  malformations 64D 

Other  circulatory/respiratory  anomalies 
(Specify)    65  D 

Rectal  atresia/stenosis 66D 

Tracheoesophageal  fistula/Esophageal 

atresia 67D 

Omphalocele/Gastroschisis  68D 

Other  gastrointestinal  anomalies 

(Specify)    69D 

Malformed  genitalia 70 1 

Renal  agenesis 71 D 

Other  urogenital  anomalies 
(Specify)    72D 

Cleft  lip/palate 73D 

Polydactyly/Syndactyly  74D 

Limb  reduction(s) 75D 

Club  foot 76D 

Diaphragmatic  hernia 77D 

Other  musculoskeletal/integumental  anomalies 
(Specify)    78D 

Down's  syndrome 79D 

Other  chromosomal  anomalies 
(Specify)    80  D 

Other  81 D 

(Specify) 
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TECHNIQUES  FOR  COMBINING  QUALITATIVE  AND  QUANTITATIVE  DATA 
TO  IMPROVE  THE  VALIDITY,  RELIABILITY,  AND  COMPARABILITY  OF  HEALTH  SURVEYS 

Carol  N.  D'Onofrio,  Northern  California  Cancer  Center 
Susan  L.  Stewart,  Rena  J.  Pasick,  Joyce  A.  Bird,  Joan  R.  Bloom 


Rapid  changes  in  the  racial  and  ethnic 
composition  of  the  U.S.  population  are 
accompanied  by  increasing  linguistic  and 
cultural  heterogeneity.   Due  to  heavy  migration, 
the  San  Francisco  Bay  Area  enjoys  a  particularly 
rich  mix  of  peoples  from  diverse  racial,  ethnic, 
cultural,  and  linguistic  backgrounds. 

As  public  health  investigators  who 
regularly  survey  this  population,  we  began  to 
question  the  extent  to  which  health  data 
collected  from  multi-ethnic,  multi-lingual 
people  living  in  the  same  geographic  region  are 
valid,  reliable,  and  comparable.   In  1992,  we 
received  a  grant  from  the  National  Center  for 
Health  Statistics  to  study  this  question. 

This  paper  briefly  describes  the  various 
qualitative  and  quantitative  research  techniques 
independently  employed  to  identify  potential 
methodological  problems  affecting  the  quality  of 
data  collected  from  five  racial/ethnic  groups: 
African  American,  Chinese,  Latina,  Vietnamese, 
and  white.   We  then  illustrate  methods  for 
combining  res\ilts  of  qualitative  and 
quantitative  data  analysis  to  determine  whether 
discrepancies  observed  represent  real  population 
differences  or  bias  due  to  methodological 
problems.   We  also  discuss  some  of  the 
techniques  we  are  using  to  isolate 
methodological  difficulties  and  to  test 
potential  solutions  to  these  problems. 

We  do  not  presume  to  have  definitive 
answers  to  the  issues  raised  by  this  work,  but 
we  have  identified  some  promising  directions  for 
their  continuing  study. 

Background 

Our  project  is  a  methodological 
investigation  coordinated  with  three 
intervention  studies  funded  by  the  National 
Cancer  Institute.   All  three  intervention 
studies  are  aimed  at  improving  breast  and 
cervical  cancer  screening  among  low- income  women 
and  ethnic  minorities  living  in  the  San 
Francisco  Bay  Area.   Each  of  these  studies  is 
conducted  by  a  mult i- disciplinary  team  with 
strong  representation  from  the  targeted 
racial/ethnic  groups  and  close  ties  to  their 
respective  communities .   Important  for  our 
purposes,  all  three  studies  also  collect  data 
from  these  communities  in  pre-  and  post- 
intervention  surveys. 

Since  two  of  the  three  projects  were 
initiated  one  year  before  our  NCHS  project  was 
funded,  our  work  with  these  studies  was 
retrospective.   The  Breast  and  Cervical  Cancer 
Intervention  Study  (BACCIS)  conducted  household 
interviews  with  low- income  women  living  in 
census  tracts  near  health  department  clinics  in 
two  counties.   The  baseline  survey  in  1992-93 
yielded  1,599  completed  interviews,  including 
475  with  African  Americans,  279  with  Chinese, 
230  with  Latinas,  500  with  whites,  and  115  with 
women  from  other  racial  and  ethnic  groups. 
Chinese  respondents  completed  the  survey  in 
English,  Cantonese,  or  Mandarin,  and  Latina 
respondents  were  offered  a  choice  of  Spanish  or 
English. 

During  the  same  period,  the  Vietnamese 
Prevention  Study  (VPS)  conducted  baseline 


telephone  interviews  with  933  Vietnamese 
women  living  in  San  Francisco  and  Orange 
Counties.   All  of  these  interviews  were 
conducted  in  Vietnamese. 

The  Pathways  project  was  initiated  in  the 
same  year  as  our  NCHS  investigation  and  thus  we 
could  track  each  stage  of  survey  development  and 
implementation  prospectively.   Pathways  is  a 
multi- component  program  project  consisting  of 
independent  community  intervention  studies  with 
Latinas  and  Vietnamese  women,  another 
intervention  study  in  a  hospital  Emergency 
Department  that  serves  mostly  African  Americans, 
a  population  survey  of  Chinese  Americans,  and  a 
shared  cross-cultural  core. 

Working  through  this  core,  common  questions 
were  developed  for  use  in  surveying  African 
American,  Chinese,  Latina,  Vietnamese,  and  white 
women.1   Since  breast  and  cervical  cancer 
screening  rates  are  high  among  African  American 
and  white  women  in  the  San  Francisco  Bay  Area, 
these  populations  were  surveyed  for  comparison 
purposes  only.   In  summer  and  fall  1993,  data 
were  obtained  from  1,601  Latinas,  645 
Vietnamese,  775  Chinese,  602  African  American, 
and  605  white  women.   As  in  BACCIS,  Chinese  and 
Latina  participants  in  Pathways  surveys  were 
offered  a  choice  of  language. 

Preliminary  Methods 

The  first  step  in  our  NCHS  project  was  to 
distinguish  variables  that  seemed  to  work  well 
across  populations  from  those  that  appeared 
troublesome.   We  approached  this  task 
independently  through  both  qualitative  and 
quantitative  routes. 

On  the  qualitative  side,  we  interviewed 
investigators  who  developed  the  survey 
questionnaire  for  each  of  the  three  intervention 
studies,  those  who  translated  and  back- 
translated  the  various  surveys  into  Spanish, 
Cantonese,  Mandarin,  and  Vietnamese,  and  the  bi- 
lingual survey  workers  who  actually  collected 
data  in  English  or  one  of  these  other  languages. 


Prior  to  the 
of  these  people  to 
respective  survey 
varied  with  their 
for  example,  were 
difficult  it  was  t 
or  difficult  they 
from  their  respect 
it,  and  the  questi 
own  particular  cul 
encouraged  on  the 


se  interviews,  we  asked  each 
rate  each  question  on  their 
on  three  dimensions  which 
research  role.  Survey  workers, 
asked  to  rate  how  easy  or 
o  ask  each  question,  how  easy 
thought  it  was  for  respondents 
ive  ethnic  groups  to  answer 
on' s  appropriateness  in  their 
ture .   Written  comments  were 
rating  form. 


Interviews  were  organized  around  a 
systematic  review  of  each  survey  instrument, 
with  in-depth  discussion  of  questions  identified 
as  troublesome  in  written  rating  forms  and 
comments.   Pretest  results  and  subsequent 
revisions  were  also  discussed  with  those  who 
designed  and  translated  the  questionnaire  for 
each  survey. 

Resulting  qualitative  data  enabled  us  to 
identify  problematic  questions  and  variables 
within  each  survey  and  ethnic/racial  group. 
Once  this  was  done,  results  were  consolidated  to 
identify  variables  that  were  troublesome  across 
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surveys  and  racial/ethnic  groups. 

While  this  work  was  in  progress,  the 
principal  investigators  and  statisticians  for 
each  intervention  study  proceeded  with 
quantitative  analysis  of  their  survey  data. 
They  used  standard  techniques  for  examining  the 
distribution  of  the  data,  bivariate,  and  then 
multi-variate  relationships.   Variables  with 
missing  data  from  many  respondents  and  those 
that  did  not  behave  as  expected  in  analysis  were 
independently  identified  as  potentially 
problematic . 

Our  NCHS  project  enabled  us  to  have 
interviewers  call  back  a  10%  random  sample  of 
respondents  from  the  five  ethnic  groups  surveyed 
for  Pathways.   To  check  reliability  of 
responses,  the  brief  callback  questionnaire 
repeated  four  questions  from  the  original 
Pathways  survey.   Respondents  also  were  asked 
whether  they  found  certain  questions 
embarrassing,  offensive,  or  difficult.   Then 
they  were  asked  how  likely  most  people  were  to 
answer  these  questions  truthfully.   These  data 
provided  additional  insight  into  issues  related 
to  a  few  specific  questions  identified  as 
troublesome  early  in  our  investigation. 


Income i 


A  Troublesome  Variable 


Our  work  with  survey  questions  on  income 
illustrates  many  of  the  ways  we  have  combined 
qualitative  and  quantitative  data  in  analysis. 

Our  qualitative  work  with  the  BACCIS  and 
VPS  surveys  quickly  identified  income  as  a 
variable  difficult  to  measure  within  and  across 
racial/ethnic  groups.   In  fact,  this  variable 
posed  so  many  problems,  that  Pathways 
investigators  decided  to  collect  income  data  in 
two  ways.   Fortuitously  for  our  NCHS  project, 
this  decision  embodied  an  important  technique 
for  improving  health  surveys --the  testing  of 
alternative  questions  forms.   However,  this 
natural  experiment  had  a  limitation:  the  order 
in  which  questions  were  presented  was  not 
randomly  varied. 

The  first  Pathways  income  question  aimed 
simply  to  determine  whether  the  respondent's 
family  lived  above  or  below  the  poverty  level. 
This  was  preceded  by  a  question  inquiring  about 
family  household  size.   The  interviewer  then 
used  a  chart  to  determine  the  poverty  line  for  a 
family  of  that  size  and  simply  asked  the 
respondent  whether  the  combined  monthly  income 
of  the  family  was  more  or  less  than  that  amount. 
The  second  question  simply  asked  respondents  to 
estimate  the  approximate  total  income  of 
household  members  for  the  last  tax  year. 

These  questions  came  near  the  end  of  the 
interview  and  were  prefaced  by  a  statement 
acknowledging  that  it  is  difficult  to  estimate 
income,  but  that  this  information  is  important 
in  understanding  the  need  for  health  care 
assistance  in  the  respondent's  ethnic  community. 
Assurances  about  the  confidentiality  of 
information  provided  in  the  interview  were 
repeated. 

Qualitative  Data  on  Income  Questions 

Qualitative  data  from  the  Pathways  survey 
revealed  many  problems  with  both  of  these 
questions.   Across  ethnic  groups  we  were  told 
many  people  did  not  like  to  reveal  their  income. 
They  did  not  understand  why  this  information  is 
relevant  to  studying  health,  and 
they  worried  about  how  the  data  would  be  used. 


We  also  were  told  that  Chinese  would 
underestimate  their  income  and  that  Vietnamese 
would  not  provide  truthful  responses.   Those  who 
interviewed  Latinas  complained  that  the  income 
questions  were  difficult  because  people  had  to 
stop  and  think.   Determining  household  size  for 
the  poverty  question  was  a  complex  task  for 
immigrants  who  doubled  up  with  other  families  to 
save  money,  and  for  all  respondents  whose  older 
children  and  extended  family  members  sometimes 
lived  with  them. 

Interviewers  from  all  racial/ethnic  groups 
did  not  like  the  income  questions  because  they 
interfered  with  respondent  rapport  and  the  pace 
of  the  interview.   If  respondents  refused  to 
answer  the  question  about  poverty  level, 
interviewers  also  felt  uncomfortable  asking  the 
next  question  about  estimated  family  income. 

Missing  Data  Rates 

Problems  with  the  Pathways  income  questions 
also  showed  up  as  soon  as  analysts  ran  frequency 
distributions .   Considering  both  refusals  and 
don't  know  responses,  the  missing  data  rate  for 
most  questions  was  less  than  5%,  but  missing 
data  on  the  two  income  questions  ranged  from  10- 
22%  for  all  groups  except  the  Vietnamese.   These 
results  would  have  been  difficult  to  interpret 
without  qualitative  data  from  Vietnamese 
interviewers  indicating  their  people  from  their 
community  would  "give  a  number",  but  it  would  be 
wrong . 

In  every  racial/ethnic  group,  respondents 
were  more  likely  to  answer  the  dichotomous 
question  on  poverty  status  than  the  more 
detailed  question  on  income.   As  Table  1  shows, 
respondents  who  disclosed  their  income  almost 
always  answered  the  question  on  poverty  status . 
In  addition,  a  fair  proportion  of  people  who 
would  not  estimate  their  annual  family  income 
indicated  whether  their  family  lived  above  or 
below  poverty  level.   Still,  from  4-11%  of  each 
racial/ethnic  sample  answered  neither  question. 


Table  1. 


Percentage  Answering  Poverty  Status 
and  Income  Questions 


Both 


Only     Only  Neither 
Poverty    Income 


Af rican-Amer . 

82 

8 

1 

9 

Chinese 

80 

10 

1 

8 

Latina 

76 

11 

2 

10 

Vietnamese 

93 

2 

- 

4 

White 

82 

6 

6 

11 

Therefore  we  asked,  What  are  the 
characteristics  of  people  who  do  not  provide 
information  about  income?   Why  are  they  not 
answering?   How  good  are  the  data  on  income  that 
we  did  obtain? 

Who  Doesn't  Answer  Questions  about  Income? 

Due  to  their  low  rate  of  missing  data  on 
poverty  status  and  income,  Vietnamese  were 
excluded  analyses  of  characteristics  associated 
with  missing  data  on  these  variables.   In  all 
other  groups,  older  respondents  and  those  with 
less  than  12  years  of  education  were  least 
likely  to  answer  the  income  question.  Blacks  and 
Chinese  with  five  or  more  in  their  households 
also  were  likely  to  have  missing  income  data. 
(See  Table  2) . 

Age  and  household  size  had  the  same 
relationship  to  missing  data  on  poverty  level. 
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However,  education  was  related  to  missing  data 
on  poverty  level  only  for  Hispanics . 

These  quantitative  results  were  consistent 
with  our  qualitative  data.   For  example,  we  were 
told  that  older  Chinese  women  live  with  an  adult 
son  and  they  don't  know  or  care  how  much  he 
makes.   Among  African  Americans,  older  women  on 
social  security  and  widows  were  most  suspicious 
about  the  income  questions.   Some  African 
American  respondents  told  interviewers  it  was 
impolite  to  ask  about  income. 


Table  2.     Variables  Related  to  Missing  Data  on 
Annual  Family  Income 

AgeEducationHousehold 
Size 


Af rican-Amer . 
Chinese 
Latina 
White 


0.001  0.01 

0.001  0.001 

0.001  0.001 

0.001  0.01 


0.001 
0.004 


Callback  Interviews 

Callbacks  to  a  10%  random  sample  of  persons 
from  each  racial /ethnic  group  who  had  completed 
the  Pathways  questionnaire  provided  additional 
insight  on  why  some  people  did  not  provide  data 
on  income.   A  sizable  proportion  of  respondents 
considered  these  questions  embarrassing, 
offensive,  and  difficult.   As  Table  3  shows, 
Chinese,  Latina,  and  Vietnamese  respondents  were 
much  more  likely  to  consider  these  questions 
difficult  than  embarrassing  or  offensive. 


third  of  the  Latinas  and  Vietnamese  thought  that 
most  people  were  not  very  or  not  at  all  likely 
to  be  truthful.   Notably,  one-third  of  the 
Vietnamese  respondents  called  back  chose  not  to 
answer  this  question. 

Table  4  shows  the  distribution  of 
respondents  who  thought  people  would  not  answer 
income  questions  truthfully  by  income  reported 
in  the  Pathways  survey.   Although  the  proportion 
of  African  Americans,  Chinese,  and  whites  who 
felt  people  would  be  untruthful  did  not  vary  by 
income,  most  persons  in  these  groups  who  said 
they  thought  people  would  be  untruthful  reported 
incomes  above  the  poverty  line.   These  data 
coincide  particularly  with  qualitative  data  we 
later  obtained  from  focus  groups  indicating  that 
African  Americans  may  be  embarrassed  to  admit 
low  incomes . 

In  contrast,  most  of  the  Latinas  and 
Vietnamese  who  said  that  people  would  be 
untruthful  in  reporting  income  themselves 
reported  low  incomes  and  poverty  status. 
Qualitative  data  revealed  that  concerns  about 
immigration  status  and  eligibility  for  welfare 
benefits  may  motivate  under -reporting  of  income 
by  these  groups .   Such  analyses  can  help  to 
estimate  the  direction  of  bias  in  problematic 
data;  however,  we  have  not  yet  resolved  ethical 
issues  raised  by  our  findings. 


Table  4 .     Percentage  Not  Likely  to  Answer 

Questions  on  Income  Truthfully  by 
Reported  Income  and  Poverty  Status 


$20K 


$20K 


Below    Above 
Poverty   Poverty 


Table  3 .     Percentage  Saying  Income  Questions 
Very  or  Somewhat 


Emba 

rrassi 

ng 

Offensive 

Difficult 

Af rican-Amer . 

17 

20 

24 

Chinese 

27 

26 

49 

Latina 

14 

9 

31 

Vietnamese 

19 

22 

38 

White 

28 

29 

25 

We  cross -tabulated  these  data  from  the 
callback  interviews  with  the  information  on 
income  that  these  respondents  provided  in  the 
Pathways  survey.   Results  revealed  that  Chinese 
and  whites  who  had  higher  incomes  and  were  above 
the  poverty  line  were  more  likely  than  those 
with  low  incomes  to  find  the  income  questions 
difficult.   The  reverse  situation  was  found  for 
Latinas  and  Vietnamese:   those  with  lower 
incomes  and  who  were  below  the  poverty  line  were 
most  likely  to  consider  the  income  questions 
difficult . 

The  African  American  data  were  not  easy  to 
interpret.   Respondents  reporting  annual  family 
incomes  less  than  $20,000  were  more  likely  to 
find  the  income  questions  difficult  than  those 
with  higher  incomes  (13%  v  5%) .   On  the  other 
hand,  African  Americans  below  the  poverty  line 
were  less  likely  to  find  the  income  questions 
difficult  than  those  above  the  poverty  line  (4% 
v  19%) .   These  puzzling  results  may  be  a 
function  of  the  income  cut -point  used  in  these 
analyses . 

In  the  callback  interviews,  respondents 
also  were  asked  how  likely  most  people  would  be 
to  answer  questions  about  income  truthfully. 
Compared  to  14%  of  whites,  over  20%  of  the 
African  Americans  and  Chinese  and  about  one- 


Af rican-Amer . 

7 

8 

1 

17 

Chinese 

10 

10 

2 

19 

Latina 

17 

4 

18 

9 

Vietnamese 

26 

3 

19 

11 

White 

4 

4 

3 

8 

Further  analysis  of  the  callback  data 
promises  to  reveal  more  about  the  validity, 
reliability,  and  comparability  of  responses  to 
the  income  questions.   We  have  found,  for 
example,  that  answering  questions  about  income 
truthfully  does  not  appear  to  be  related  to  the 
language  of  the  interview. 

Reliability  of  Data  on  Income 

As  we  uncovered  problems  with  Pathways 
survey  data  on  income,  concern  mounted  about 
implications  for  data  analysis  in  the  Pathways 
program  project.   Should  data  on  annual  family 
income  and  poverty  status  be  ignored  in  analysis 
or  can  these  variables  contribute  to 
understanding  issues  in  utilization  of  cancer 
screening  by  the  five  ethnic  groups? 

To  investigate  this  issue,  we  checked  the 
reliability  of  data  on  income  and  poverty  status 
through  a  series  of  cross-tabulations  and 
correlations.   As  Table  5  illustrates, 
respondents  with  annual  family  incomes  under 
$20,000  were  much  more  likely  to  be  below  the 
poverty  level  than  were  those  with  higher 
incomes.   This  result  provides  some  evidence  of 
data  reliability. 

Again  qualitative  data  combined  with 
additional  quantitative  analysis  helped  to 
explain  between-group  differences  observed:   due 
in  part  to  larger  household  sizes,  Hispanics  and 
Vietnamese  were  much  more  likely  to  have  incomes 
below  the  poverty  level  than  were  African 
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Americans,  Chinese,  and  particularly,  whites. 


Table  5 . 


Percentage  Below  the  Poverty  Level 
by  Annual  Family  Income 


$20K 


Af rican-Amer . 

33 

Chinese 

37 

Latina 

71 

Vietnamese 

74 

White 

21 

>  $20K 

3 

5 

13 

18 

3 


about  income  should  be  clearly  explained,  (2) 
alternative  ways  to  collect  income  data  should 
be  tested,  (3)  a  specific  code  for  missing  data 
should  be  included  in  data  sets,  and  (4)  the 
reliability  of  income  data  should  be  checked 
through  respondent  callbacks,  as  well  as  through 
cross-tabulations  and  correlations  with  other 
related  variables.   Finally,  (5)  throughout  data 
analysis  it  is  important  to  be  aware  of  possible 
biases  and  not  to  assume  that  data  on  income  are 
automatically  valid,  reliable,  and  comparable 
across  racial/ethnic  groups. 


Comparing  the  percent  of  respondents 
reporting  that  they  had  private  health  insurance 
by  reported  annual  family  income  provided 
additional  evidence  that  the  latter  variable  is 
reliable.   Table  6  shows  that  in  all  groups, 
respondents  with  a  yearly  family  income  of 
$20,000  or  more  were  much  more  likely  to  have 
private  health  insurance  than  were  respondents 
with  lower  family  incomes.   However,  large 
disparities  were  observed  in  the  proportion  of 
each  racial/ethnic  group  with  private  health 
insurance.   Most  Chinese  and  whites  had  private 
insurance  even  if  their  family  incomes  were  less 
than  $20,000  per  year.   In  contrast,  only  one- 
third  of  higher  income  Vietnamese  had  private 
insurance . 


Table  6. 


Percentage  with  Private  Health 
Insurance  by  Annual  Family  Income 


$20K 


$20K 


Af rican-Amer . 

35 

Chinese 

66 

Latina 

23 

Vietnamese 

7 

White 

54 

91 
91 
79 
35 
85 


We  corroborated  these  findings  by 
conducting  similar  analyses  with  data  from 
BACCIS  and  the  VPS.   In  the  BACCIS  data,  for 
example,  income  treated  as  a  continuous  variable 
was  strongly  and  significantly  correlated  with 
having  private  health  insurance  in  each 
racial/ethnic  group.   Table  7  shows  that  these 
correlations  were  significantly  larger  for 
African  Americans  and  Chinese  than  for  whites, 
but  all  correlations  were  in  the  same  direction. 


Table  7, 


BACCIS:   Correlation  of  Income  with 
Private  Health  Insurance 


Table  8.     BACCIS:   Correlation  of  Income  with 
Education 


0.001 
0.  001 
0.  001 
0.001 
0.001 


Other  Illustrations 

Other  data  illustrate  two  additional  ways 
in  which  we  have  combined  qualitative  and 
quantitative  analysis  to  examine  the  validity, 
reliability,  and  comparability  of  data  gathered 
from  multi-ethnic  populations  in  health  surveys. 


Af rican-Amer . 

0 

31 

Chinese 

0 

459 

Other  Asian 

0 

336 

Latina 

0 

372 

White 

0 

417 

Conducting  fac 
group  can  be  a  usef 
the  BACCIS  data  for 
barriers  to  medical 
for  all  racial/ethn 
loaded  in  the  same 
each  group .  Howeve 
do  you  feel  your  do 
your  concerns  about 
for  Chinese,  but  on 
Our  qualitative  dat 
translators  conside 
difficult  to  expres 
We  dropped  the  item 
of  the  resulting  sc 
groups . 


tor  analyses  by  racial/ethnic 
ul  technique.   We  did  this  in 
eight  items  concerning 
care.   Three  factors  emerged 
ic  groups,  and  seven  items 
way  on  these  factors  within 
r,  one  question,  "How  often 
ctor  or  nurse  understands 
health?",  loaded  on  Factor  3 
Factor  1  for  other  groups, 
a  revealed  that  Chinese 
red  this  item  one  of  the  most 
in  Mandarin  and  Cantonese . 
and  improved  the  reliability 
ales  across  all  racial/ethnic 


Table  9  shows  the  Cronbach  alphas  for  the 
three  factors  after  the  troublesome  question  was 
removed.   The  communication  factor  is  very 
reliable  with  alpha  near  .8  for  all 
racial/ethnic  groups.   The  financial  factor  is 
also  very  solid.   However,  the  availability 
factor  is  less  reliable,  especially  for  African 
Americans,  and  must  be  viewed  with  caution.   Our 
next  step  will  be  to  correlate  each  of  these 
factors  with  variables  related  to  factor 
content . 


Af rican-Amer . 

0 

481 

Chinese 

0 

519 

Other  Asian 

0 

508 

Latina 

0 

398 

White 

0 

369 

0.001 
0.  001 
0.  001 
0.001 
0.001 


Table  9. 


Similarly,  as  shown  in  Table  8, 
correlations  between  reported  income  and 
education  were  significant  and  strong  within 
each  of  the  groups  surveyed  by  BACCIS.   These 
results  provide  further  evidence  that,  despite 
its  problems,  income  is  a  reasonably  reliable 
variable . 

This  does  not  mean,  of  course,  that  we 
should  ignore  the  issues  discovered.   We 
conclude  from  our  qualitative  and  quantitative 
data  analyses  that  (1)  reasons  for  inquiring 


BACCIS:   Barriers  to  Medical  Care, 
Cronbach' s  Alpha  for  Three  Factors 


Communication 

Financia 

1 

Availability 

African- 

-Amer . 

0.  78 

0.80 

0.36 

Chinese 

0.79 

0.78 

0.48 

Latina 

0.80 

0.89 

0.62 

White 

0.78 

0.89 

0.53 

Total 

0.79 

0.84 

0.51 

Another  technique  is  to  compare 
reliabilities  by  racial/ethnic  group.   Table  10 
shows  the  percentage  of  responses  to  three 
questions  from  callback  interviews  that  agreed 
with  the  responses  given  at  the  time  of  the 
Pathways  survey.   Agreement  was  high  on  the 
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question,  "Have  you  ever  had  a  mammogram?"   This 
is  a  factual  question  about  a  clearly  defined 
event  that  can  be  answered  simply  by  saying 
"yes"  or  "no" . 

The  other  two  questions  had  numeric 
responses  and  showed  significant  differences 
among  groups  when  exact  concordance  was 
required.   Using  other  measures,  "times  you  saw 
a  doctor  in  the  past  year"  still  showed 
significant  variation  between  groups,  but  "years 
in  the  U.S."  showed  generally  good  agreement 
and  groups  did  not  significantly  differ  from 
each  other.   Our  qualitative  data  revealed  that 
people  have  difficulty  remembering  the  number  of 
times  they  saw  the  doctor  in  the  past  year.   The 
year  of  immigration  is  memorable,  but 
calculating  total  years  in  the  U.S.  can  be 
challenging  during  an  interview  and  errors 
apparently  are  made. 


Table  10.    Percent  Agreement  between  Pathways 
Survey  and  Callbacks  on  Three 
Questions 

Ever  HadTimes  Saw  Years  in 
Mammogram     M.D.     U.S. 


African-Amer . 

96 

40 

Chinese 

95 

33 

82 

Latina 

95 

50 

78 

Vietnamese 

89 

27 

51 

White 

97 

42 

Summary 

In  summary,  the  validity,  reliability,  and 
comparability  of  health  survey  data  collected 
from  multi-ethnic,  multi-lingual  populations 
should  not  be  taken  for  granted. 

Some  qualitative  methods  that  investigators 
can  easily  use  to  check  the  quality  of  their 
data  are : 

•  Carefully  examine  pretest  results.   In  the 
rush  to  get  surveys  into  the  field,  pretest 
findings  are  often  used  only  to  revise  questions 
and  then  forgotten. 

•  Ask  investigators,  translators,  and  survey 
workers  to  rate  and  comment  on  questions. 

•  Debrief  survey  workers.   After  completing 
data  collection,  interviewers  have  a  wealth  of 
information  about  which  questions  do  and  do  not 
work  well  and  why. 

Other  qualitative  methods  used  in  our  NCHS 
investigation  include  focus  groups2  and  think 
aloud  interviews3  conducted  with  low-income 
women  from  the  five  racial /ethnic  groups  in 
their  native  language.   We  also  tracked  the 
evolution  of  questions  in  the  Pathways  survey 
instrument,  keeping  notes  about  the  reasons 
draft  items  were  modified  or  discarded.   Later 
we  interviewed  Pathways  core  investigators  from 
the  relevant  racial/ethnic  groups  to  verify  and 
discuss  the  cultural,  linguistic,  or  other 
issues  that  affected  their  shaping  of  survey 
questions.    Such  techniques  have  not  been 
discussed  in  this  paper  because  they  require 
time  and  resources  generally  unavailable  to 
health  survey  researchers. 

Quantitative  methods  that  are  readily 
applied  include  checking  missing  data  rates  and 
analyzing  data  by  race/ethnicity,  as  well  as  by 
language  of  interview,  country  of  birth  and 
other  variables  such  as  age  and  education  that 


may  affect  sub-group  responses. 

Quantitative  methods  that  require 
additional  time  and  resources  include  callbacks 
to  a  random  sample  of  survey  respondents  to 
check  the  reliability  of  responses  to  particular 
questions  and  to  obtain  data  on  attitudes  toward 
troublesome  survey  items.   Comparing  alternative 
forms  of  questions  to  gather  data  on  the  same 
variable  is  also  a  valuable  technique. 

While  both  qualitative  and  quantitative 
methods  used  independently  can  provide  important 
insight  into  the  quality  of  health  survey  data, 
we  have  found  that  these  approaches  complement 
each  other  in  (1)  identifying  troublesome 
variables;  (2)  checking  the  validity, 
reliability,  and  comparability  of  data  across 
racial/ethnic  groups;  and  (3)  pinpointing 
particular  methodological  problems.   Combining 
qualitative  and  quantitative  approaches  also  (4) 
increases  sensitivity  to  potential  bias  in  the 
data.   And  both  approaches  have  important 
contributions  to  make  in  (5)  identifying  and 
testing  better  survey  questions  and  data 
collection  methods. 

We  find  this  work  exciting  and  encourage 
others  who  conduct  qualitative  and  quantitative 
analyses  such  as  we  have  described  today  to 
disseminate  their  results  to  the  research 
community.   To  improve  the  validity, 
reliability,  and  comparability  of  health  data 
obtained  from  multi-ethnic  populations,  results 
of  methodological  studies  conducted  with 
different  populations  and  in  different  regions 
of  the  country  also  need  to  be  shared  and 
compared. 
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LINKING  DATA  SETS  FOR  STUDY  OF  TREATMENT  VARIATION  AMONG  MINORITY  POPULATIONS 

Selma  C.  Kunitz,  KAI ;  Matthew  Jaro,  MatchWare;  Gene  Therriault  NY  statf* 
Department  of  Public  Health;  Rene  Kozloff,  KAI ;    Harvey  Schwartz   IhSr 


The  primary  objectives  of  this 
record  linkage  methodology  project 
sponsored  by  AHCPR  through  contract 
282-94-2005  are  to:l)  link  two 
patient  level  related  data  sets  that 
contain  racial  and  ethnic 
descriptors;  and  2)  assess  the  value 
of  the  linked  data  to  address  medical 
effectiveness  research  questions  that 
focus  on  the  quality,  effectiveness, 
and  outcomes  from  care  for  minority 
populations.  Activities  include: 

•  Identification  of  suitable  health 
related  data  sets; 

•  Linkage  of  these  data  sets; 

•  Assessment  of  the  quality  and 
utility  of  the  data  with  respect 
to  addressing  health  services 
research  on  minority  populations; 

•  Creation  of  a  user- friendly  manual 
that  provides  a  step-by-step 
approach  to  linkage  methodology 
for  use  by  researchers;  and 

•  Documentation  of  the  project 
through  a  report  that  describes 
the  data  accuracy  and  reliability, 
problems  and  their  resolution, 
project  steps,  and  recommendations 
for  future  related  endeavors. 

The  linkage  software  used  for  this 
project  is  MatchWare  Technology 
Incorporated' s  (MTI)  AUTOMATCH.  It  is 
generalized  to  be  consistent  with 
good  statistical  practice.  Data 
management,  clerical  review,  linked 
record  extraction,  unduplication, 
blocking,  multi-pass  matching  and 
frequency  analysis  are  all  provided 
in  both  personal  computer  and 
mainframe  environments.  It  is  a 
robust  system  with  the  following 
features : 

•  PC  file  size  limitations  -  to 

avoid  problems  associated  with 
storing  multiple  copies  of  large 
data  files,  or  the  slow  speed 
associated  with  relational  data 
base  management  systems,  the 
record  linkage  software  provides 
blocking  by  an  indexing  system 
rather  than  sorting.  Multiple-pass 
matching  is  handled  through  the 
creation  of  residual  pointer  files 
keep  track  of  records  matched  on 
previous  passes. 

•  Questionable  links  -  to  handle 
uncertain  matches,  an  individual 
reviews  the  linked  pairs  to 


establish  decision  cut-off 
thresholds.  The  clerical  review 
system  maintains  a  history  file  so 
that  the  same  records  do  not  have 
to  be  continually  reexamined  if 
the  matcher  is  re -run. 

•  Duplicate  records  -  all  duplicate 
records  are  identified  on  all 
files  through  a  report  generator 
and  file  extraction  program. 

Selection  of  Data  Sets 

We  received  approval  from  the 
State  of  New  York  to  utilize  the 
Statewide  Planning  and  Research 
Cooperative  System  (SPARCS)  files 
which  contain  all  acute  hospital 
discharge  and  claims  data,  the  SPARCS 
ambulatory  surgical  files,  and  the 
Cardiac  Surgery  Reporting  System 
(CSRS)  files  which  are  a  research 
data  set.  Since  the  project  was 
initiated  in  1994,  we  requested  files 
for  the  1991,  1992,  and  1993  time 
periods.  Figure  1  summarizes  the  size 
of  the  original  data  files. 

While  the  discharge  and  claims 
files  have  been  discrete  files,  the 
State  of  New  York  will  combine  the 
data  beginning  in  1995  and  thus, 
provided  them  in  a  combined  format . 
The  ambulatory  surgery  files  were  not 
segregated  by  year  and  contain 
slightly  more  than  two  million 
records.  The  CSRS  data  files  are  also 
summarized  by  year  and  contain  a 
considerably  smaller  number  of 
records  because  of  the  focus  of  the 
records.  Hannan  et  al,    1990;  Hannan 
et  al,    1992  articles  describe  the 
utility  of  these  data  for  medical 
effectiveness  studies. 

OBTAINING   THE  DATA 

The  process  of  obtaining  data 
sets  always  takes  longer  than 
anticipated.  We  began  the  request  for 
the  data  sets  in  June,  1994  and 
received  one  set  in  January,  1995  and 
another  set  in  April,  1995.  Some  of 
the  issues  we  confronted  in  the  data 
acquisition  process  are  as  follows: 

•  Identifying  Responsible  Source  - 

identifying  the  responsible  person 
for  the  SPARCS  data  sets  which  are 
administrative  data  sets  was  not 
an  issue.  However,  identifying  the 
responsible  person  for  the  CSRS 
research  data  sets  was  a  time 
consuming  task.  It  took 
approximately  two  months  to 
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identify  the  person  responsible 
for  approving  data  set  use  and 
almost  daily  telephone  calls  to 
ensure  that  we  actually  received 
the  data. 

•  Field  Size  Differences  -  The  MRN 
in  the  SPARCS  DDA/UBF  file  is  10 
characters  and  was  encrypted  as 
such.  However,  the  MRN  in  the 
Ambulatory  Surgery  file  is  defined 
as  17  characters  (the  first  10 
characters  actually  contained  the 
MRN,  the  last  seven  characters 
were  spaces.)   When  an  initial 
match  was  attempted  between  the 
DDA/UBF  and  the  Ambulatory  Surgery 
files,  not  one  set  of  records 
matched.  The  problem  was  resolved 
by  recreating  the  Ambulatory 
Surgery  File  using  only  the  first 
10  characters  of  the  MRN. 

•  Documentation  -  although  each  of 
the  data  sets  was  accompanied  by 
documentation,  it  was  not  always 
current  and  did  not  reflect  the 
actual  contents  of  the  data  sets. 

RESEARCH  QUESTION 

To  focus  the  project  a  suitable 
research  question  was  identified  that 
relates  risk  factors,  treatment  and 
outcome  of  cardiovascular  disease  to 
minority  status  as  follows: 

Are  the  racial/ethnic  differences 
in  mortality  and  morbidity  from 
coronary  heart  disease  related  to 
racial  and  ethnic  differences  in 
treatment? 

The  working  hypothesis   is  that 
minorities  are  less  likely  to  receive 
surgical  treatment  for  coronary 
artery  disease  and  therefore  as  a 
group,  experience  higher  incidence  of 
cardiovascular  morbidity  and 
mortality  than  the  majority  U.S. 
population.  The  original  research 
plan  suggested  the  use  of  ICD-9  codes 
410  -  414.  The  low  yield  on  initial 
matches,  however,  indicated  that  we 
needed  to  expand  these  codes  to 
obtain  a  more  complete  match  between 
the  SPARCS  and  CSRS  files.  The  range 
was  subsequently  expanded  to  all 
diseases  of  the  circulatory  system 
(390xx-459xx)  and  increased  the 
number  of  potential  subjects  as  shown 
in  Figure  2 . 
DATA  PREPARATION 

Encryption  -  MRN,  Admission 
numbers,  and  Physician  License 
numbers  were  encrypted  to  ensure 
privacy.  They  were  encrypted  using 
the  same  rules  across  all  files  to 
allow  matching  and  longitudinal 
analysis.  However,  the  encryption 


process  does  degrade  the  efficiei 
of  the  matching  softv/are.  The 
matching  software  used  in  this  study 
can  take  into  account  slight 
differences  between  identifiers  such 
as  transposition  of  characters  and 
adjust  the  match  for  them.  For 
example,  if  MRNs  were  off  by  one 
number  because  of  a  typographical 
error  and  all  the  rest  of  the 
information  on  the  records  used  in 
the  match  was  the  same,  the  software 
would  treat  this  as  a  typographical 
error  and  consider  the  records  to  be 
matches.  However,  the  encryption 
process  takes  two  MRNs  with 
transposed  numbers  and  converts  them 
to  two  totally  different  numbers. 
Therefore,  the  software  benefits  in 
this  area  are  lost . 

Data  Conversion  for  Linkage  - 

The  Automaton  record  linkage  system 
for  this  project  was  implemented  on  a 
high-performance  personal  computer. 
The  system  accepts  patient  level  data 
as  DOS   files  consisting  of  fixed 
length  records  with  ASCII   characters. 
Cases  and  variables  required  for 
linkage  were  selected  and  converted 
to  this  format  by  project  staff  using 
SAS   and  C  data  manipulation  programs 
developed  on  NIH  and  internal 
computer  systems  for  these  purposes. 
In  an  effort  to  reduce  the  potential 
for  false  matches,  as  well  as 
minimize  required  disk  storage  space 
and  linkage  processing  times,  only 
those  cases  and  variables  that  were 
expected  to  participate  directly  in 
the  linkage  process  were  included. 
For  ambulatory  surgery  and  inpatient 
records,  only  cases  with  cardiac 
related  diagnoses  were  included. 

Cardiac  related  diagnoses  were 
defined  as  any  diagnosis  (admitting, 
principal,  or  any  other)  of  ischemic 
heart  disease  (ICD-9  code  in  the 
expanded  range . 

Person  and  Event  Identifiers. 
The  goal  of  the  linkage  process  is  a 
longitudinal  treatment  history  for 
persons  with  cardiac  related 
diagnoses.   Person  and  event 
identifiers  that  appear  to  be  common 
across  two  or  more  of  the  original 
data  sources,  or  across  two  or  more 
events  within  the  same  data  system, 
were  identified  from  source  data 
descriptions.  Data  coding  differences 
prevented  direct  use  of  some  of  the 
original  data  values.  Instead,  all 
common  data  items  were  converted  to 
standardized  values. 

As  mentioned  above,  the  SPARCS 
records  for  this  project  were  created 
by  the  NY  DOH  through  deterministic 
matching  of  Discharge  Data  Abstract 
(DDA)  and  Uniform  Billing  Form  (UBF) 
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data.   Hospital  ID,  patient  medical 
record  number,  patient  admission 
number,  and  other  identifying 
variables  were  used  by  NY  DOH  staff 
to  link  separately  reported  DDA  and 
UBF  records.  Typical  final  linkage 
rates  over  the  study  period  were  near 
95%.  That  is,  approximately  5%  of 
reported  discharge  records  had  no 
matching  billing  record.   In 
addition,  approximately  5%  of 
reported  billing  records  had  no 
matching  discharge  record.  We  did  not 
attempt  to  link  any  residual  records 
using  probabilistic  linkage  methods. 

Person  and  Event  Identifiers  - 

SPARCS  variables  that  provided 
descriptive  information  about  a 
cardiac  patient  or  a  cardiac  related 
hospitalization  were  selected  and 
prepared  for  grouping  and  matching 
purposes.   Because  SPARCS  records 
were  created  by  linking  separate  DDA 
and  UBF  records,  the  records 
contained  data  from  both  of  these 
medical  data  sources  and  both  were 
used  for  matching. 

A  measure  of  SPARCS  data 
reliability  was  obtained  by  comparing 
values  of  person  and  event 
identifiers  that  were  common  to  both 
DDA  and  UBF.   Comparisons,  the 
results  of  which  are  shown  in  Figure 
3,  were  made  only  if  neither  value 
was  missing.   It  is  likely  that  the 
high  number  of  errors  for  admission 
number,  medical  record  number,  and 
physician  were  caused  by  the 
encryption  process.   For  example,  all 
of  the  physician  license  numbers 
noted  as  errors  had  a  value  of 
00120012,    which  may  be  the  encrypted 
version  of  00000000. 

The  matching  process  was 
initiated  with  the  SPARKS  and 
ambulatory  surgery  files  and  results 
are  shown  in  Figure  4 .  There  are 
several  items  of  interest  in  this 
figure.  Physician  discordance  is  very 
high  (72%) ,  perhaps  because  admitting 
physician  is  used  on  one  file  while 
surgeon  is  used  on  the  other. 
"Unexpected  agreements"  indicate 
individuals  who  had  ambulatory 
surgery  and  were  then  admitted  to  the 
hospital,  perhaps  because  of 
complications.  Figure  5  illustrates 
the  matching  process  between  the  CSRS 
files  and  UBF/DDA  files  for  1993.  The 
numbers  are  consistent  across  years. 
The  higher  the  weight  the  greater  the 
credibility  of  the  match  and  thus, 
97%  of  the  matches  weights  greater 
than  20  indicate  "good"  matches. 
Figure  6  shows  the  match  and  error 
counts  for  4  and  6  passes  that  were 
performed  to  obtain  matches  for  the 
1993  data.  In  the  85%  (6  passes)  of 
the  22,492  CSRS  records  that  were 


matched  with  the  UBF/DDA  files,  over 
79%  of  the  matched  records  had  1,  2, 
or  3  disagreements  among  fields.  As 
shown  in  Figure  7,  MRN,  attending 
physician,  and  procedure  had  high 
degrees  of  discordance,  perhaps 
because  admitting  number  is  sometimes 
used  in  place  of  MRN,  because 
attending  and  operating  physician  are 
often  interchanged  on  the  these 
files,  and  because  a  procedure  may  be 
listed  as  primary  on  one  file  but  not 
on  the  other  because  of  reimbursement 
policies . 

Since  a  primary  research 
interest  for  this  project  is  the 
difference  in  treatment  across  racial 
and  ethnic  subgroups,  consistency  of 
these  field  prior  to  matching  and 
between  matched  and  unmatched  records 
is  important.  Figures  8,  9,  and  10 
illustrate  the  consistency  of 
percentages  of  racial  and  ethnic 
minority  patients  in  the  CSRS  and 
UBF/DDF  files  pre  and  post  matching 
as  well  as  the  proportion  of  matched 
and  unmatched  records. 

These  initial  results  show  that 
the  initial  purposes  for  which  data 
are  collected  affects  the  utility  for 
linkage.  For  example,  the  UBF  files, 
which  are  used  for  billing,  are  more 
likely  to  have  accurate  admitting  and 
discharge  dates,  while  the  DDA  or 
discharge  abstracts  are  more  likely 
to  have  more  accurate  attending 
physicians.  Likewise,  the  CSRS  files 
are  more  likely  to  have  accurate 
clinical  data.  The  linkage  methods, 
which  emanate  from  the  census 
environment,  must  be  adapted  to  the 
idiosyncracies  of  health  related  data 
sets  and  thus,  the  purpose  for  which 
data  are  collected  are  very 
important.  Linkage  across  health 
related  data  sets  will  enhance  the 
value  of  the  data  sets  and  health 
related  research  will  be  aided  with 
application  of  linkage  methods. 
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FIGURE  1 :  PROJECT  DATA  FILES 
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FIGURE    2:    UNIVERSE    OF    SUBJECTS 
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FIGURE 

3 :    DISCREPENCIES    ON   UBF/DDA   MATCHED    FILE 
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F1C.URP.  4:  INITIAL  MATCH  AND  DISCORDANCE  -  UBF/DDA  WITH  AMPULATORY  SURGERY  FILE 
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FIGURE  5:  INITIAL  MATCHES  AND  DISCORDANCE 
UBF/DDA  WITH  CSRS  -  1993  FILES 
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FIGURE  6:  INITIAL  MATCHES   AND  DISCORDANCE  UBF/DDA  WITH  CSRS 
1993  FILES  -  ERROR  COUNTS 
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FIGURE  7:  INITIAL  MATCHES   AND  DISCORDANCE 
UBF/DDA  WITH  CSRS  -  1993  FILES  -  DEMOGRAPHICS 
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FIGURE  8:  RACE  ACROSS 

SUBSETS  PRIOR  TO  MATCHING 

SUBSET  AND  RACE 
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FIGURE  9: 

ETHNICITY 

-  MATCHED  AND  UNMATCHED  RECORDS  FOR  CSRS 
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FIGURE 

10:  RACE 
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